I was a developer and turned into a DBA by necessity. When I was still just a dev, I couldn't get the support I needed from the DBAs, so now I try to be the person I needed help from before.
But, I now understand the problem. At my organisation we have only 2 DBAs, and we can't give all of our time to this task.
To be a DBA is an endless battle against the entropy some devs try to create. The data model should really always be validated by someone with more knowledge and experience. Also, some young devs come up with crazy ideas (for instance, we should not use foreign keys). Others want to load insane amounts of data to process in the database, instead of preprocessing, or using a staging database. There is a continuous stream of bad ideas being implemented.
And when a bad idea gets implemented, it is really hard to undo it and it tends to just create more chaos: you need materialised views to get around bad modelling, or weird views to compensate for duplicate data, etc.
So, yes, a DBA is a really important role. With regards to it being a good job, it depends if your company takes it seriously and puts you in the development process, otherwise you'll be in for a lot of stress.
When I was a younger developer, knowledge of SQL and how your database processed data was an expectation of everyone around me in the field.
The older I get and end up around younger developers, I find that this has become a rare superpower. These days, if I’m consulting on a problem for somebody I can almost assume that it’s a database problem. Either needs to be tuned, indexes are missing, data has been corrupted or duplicated due to lack of constraints, excessive single writes or any number of other issues.
I never thought something that was always a required skill set to suddenly become rare and shocking.
> Also, some young devs come up with crazy ideas (for instance, we should not use foreign keys).
My favorite was "we shouldn't use indexes it slows down the DB" by a junior in regards to us collectively telling him to add indexes to a MondoDB collection. When we told him to research it, we meant how to do it.
>There is a continuous stream of bad ideas being implemented. And when a bad idea gets implemented, it is really hard to undo it and it tends to just create more chaos.
This describes all aspects of software development.
Textbooks advocate a split between DA data admin vs DBA database admin. The first manages the model, the second manages the tech. You sound like nobody is doing the DA role, so all misery drops on the DBA head.
Thank you for being the DBA that helps devs. I found myself similarly frustrated when I was a developer. Now I work in DevOps, and I still cannot get traditional DBAs to deliver. :(
A lot of answers here entertain the idea that DBA role are disappearing...
I would argue that it's way more accurate to say this role is actually outsourced in most organizations with all the tradeoffs that come with that strategy. I begged my former employer to subscribe to a good PostgreSQL support company. Guess what the one we picked is constantly looking to hire new DBA to sustain growth...
I was in database engineering/administration position for the last 10 year on a small company. It was exhausting because management wouldn't care to rely one guy to manage the whole data stack. It was rewarding because the CEO saw me as probably smarter than I was because I knew SQL well enough to produce outstanding value as a single individual.
Often reading HackerNews I felt like an imposter because NoSQL was the new hype and database was seen as a thing of the past... well trends reverse... and people will always need reliable systems to handle business critical data.
So maybe in a few year after few major Cloud data horror story DBA will become the new trend. No one know.
But learning how to handle data? That will always be handy no matter the underlying technology.
> this role is actually outsourced in most organizations
Or in-sourced to developers. In places I've worked it was a developer's responsibility to ensure that a DB is used properly. None of them knew the DB good enough (they need to write code and have no time to read DB docs) so the result was mixed at best.
I'd say the role is morphing. Organizations need employees who would be able to ensure consistent practices, automation, compliance on top of cloud solutions. At some point of scaling the team, having a few DB experts will increase efficiencies and ensure that practices are automated, documented, and taught to engineers compared to letting engineers discover these on their own.
Next, the DB experts help improve efficiency. It's too easy for engineers to over-provision a DB instead of rewriting the query or adding an index or two. Lastly, engineers flock to new datastores on the Cloud that may not be the best fit for the use cases at hand. So advising teams on which datastore to pick is another angle of the role.
On a big database migration a few years ago where I was working, the “solution” was to assign one of the engineers the DBA-ish role for the duration of the project. Turned out to be a bad idea, but he did his best.
If we had had anyone who was actually good at that stuff, besides the few top people who were needed elsewhere, that person would have emerged a hero.
DBA is one of those roles that needs a few years to come back into fashion. Once people realise how incredibly terrible the "new" paradigm is, it will come back in vogue.
I'm not sure if "disappearing" is the right word, but it is definitely moving towards also requiring reliability engineering / stronger than usual automation skills.
It's also a role that's thinning out, because these managed offers are good enough a lot of the time and developers are using the most common parts of the feature set on offer.
I've read a take on NoSQL that it's aimed at web developers who don't want to understand or work with a proper database. I find that in a lot of businesses, the core data is still a regular database, with NoSQL databases only used for non-critical data; it ties in to CQRS as well, with writes being done to a central database, and a read-only cache like redis being used to provide data for the client-facing applications.
If you can model data, build data warehouses, manage and maintain schemas, and help analysts write effective SQL; you'll be employed for a very long time. If your entire focus is on the system administration perspective, that career is quickly disappearing in favor of SaaS/PaaS solutions.
As a former DBA I agree. And this was the fun part of the job. The provisioning databases, installing updates, configuring, doing backups, etc., that’s all automated by any decent cloud provider. And that part is no longer necessary unless the place you work at insists on rolling their own DB instances instead of a managed provider.
Not sure if I agree about which role is disappearing when, but it does highlight the two types of DBA:
- the "Dev DBA" who makes sure ER diagrams are constructed sensibly, knows how to look at an explain plan to see where queries are going wrong, maybe even the odd stored proc (heresy!)
- the "Sys Admin DBA" who makes sure the database has enough space, rollback segments are big enough etc etc.
DBA is still a necessary job. I would say more and more so: Over my own now 25 year career in IT, the average developer seems to have less interest/knowledge in data modelling and administration. I don't think you'll find great DBA skills let alone a dedicated role in a small high velocity exciting startup - I feel they'll cloud brute force their way :). But in a large organization, a good DBA is vital. I have 2-3 on my project team and always looking for more.
It is astonishing how much a poorly tuned data request / SQL, or poorly designed data model / table, can impact performance by multiple orders of magnitude. It is also fascinating how much addition of specific index or tuning statistics can help execution. There comes a scale where throwing more hardware on performance problem is prohibitive over hiring a good DBA, as much as current thinking is that "hardware is cheap". But on a regular basis I'm seeing experienced, serious, dedicated developers create SQL that is functional correct and reads 10 billion rows into buffer in order to spit out a single row answer (sometimes of course that's necessary, but usually not :)
> Over my own now 25 year career in IT, the average developer seems to have less interest/knowledge in data modelling and administration.
Even when it's clear as day that the data model is incorrect they often just ignore it. They blame MySQL for not scaling while having ten joins per query and using ORs like they're going out of fashion. Say it can't be done in another way, even tho clearly it could.
Or say how the benchmarks for Elasticsearch are specicially crafted, you can say that when you get 30k per second and they get 50k, but when you get 50 per second and they get 50k per second it's not their benchmark that is specially crafted it's your data model is crap.
I honestly think the thing that costs most companies the most in their hosting is developers not using databases correctly and spiking the hosting costs. I literally saw one company have to spend 20,000 a month for 500 requests per second on elasticsearch.
Knowing how to data model is a skill I learned and got good at long before I was ever forced to write my first SQL statement. For all it’s power an usefulness SQL is to databases what JavaScript is to web development… some people are WAY too obsessed with using it for everything they can. And this is a bad thing. I find “DBA”s tend to reach for SQL first and can cause their own kind of chaos. From writing difficult to test code because they hand wrote SQL that only works on the production database (PostgreSQL vs in memory SQLite) to my pet peeve of constantly suggesting that a problem would be trivial to solve with stores procedures which would massively complicate (requirements completely replacing) the database migration system currently in use (because most frameworks, like Rails, Laravel and Django don’t have good built in tools for managing, versioning and updating stored procedures).
Adding a DBA to a team is like adding an opinionated senior developer, sometimes you need that experience and sometimes it’s just going to be a total fucking pain in your ass.
If you feel you need better data modelling skills in your developers, don’t hire a DBA, hire developers that know how to do data modelling.
I would argue that a developer that can't structure and query data for efficiency and integrity, is probably wreaking havoc in the rest of the systems too. You don't need a DBA, you need a better developer. Managing state is an important skill outside of DBs too.
I'm ~7 years into my professional software career and it wasn't until the last year or two I took a huge interest in the database layer, I can attribute a lot of it to Designing Data Intensive Applications.
We had a DBA about 6 years ago, he was one of the smartest guys I've ever worked with, and knew his stuff inside and out. I would recommend a DBA highly to anyone having growth issues. I honestly don't think we would have scaled without him. I learned a ton from him.
He got let go for basically having a really bad temper, and we decided not to look for a new DBA. I was sad he was gone but also glad to not be getting screamed at. It feels silly, but I've got really unresolved and mixed feelings about the whole thing. I genuinely really liked the guy, loved talking music and movies with him, but he would heel turn in an instant and just ruin your day.
Honestly wish we had a DBA, I ended up picking up a lot of his responsibilities. I'd love to have someone I could learn from. Basically, I'm getting real sick of optimizing reports.
It's important to understand why it existed in the first place. A long time ago, if you were building business software on a database at all, it was a very costly commercial database. Imagine 6 figures a year for the software licenses, the same amount again for a support contract, and a similar amount for the hardware to run it on. If a company is spending $1M+ a year on a critical piece of software infrastructure, that is very complicated in addition to being very expensive, they will usually be very open to spending more on top of that in salary to employ experts to look after it. Those experts are your DBAs.
A funny thing happened when MySQL and Postgres made databases "free": companies (irrationally, IMHO) stopped being willing to pay for experts to run their database software. That duty became a secondary responsibility of developers or operators or both, none of whom really wanted to do it or cared to learn how to do it well. There are counterexamples, sure. Some companies get big enough, and lean on their databases hard enough, that they cultivate true competence in them, but it's rare.
And then along comes AWS RDS (and similar), automating away the most tangible aspects of what DBAs did: HA, backup/restore, configuration, provisioning. There's still plenty left for DBAs to do, but it's stuff people don't understand the value of. As above, the residual DBA duties silently fall to developers and operators.
Also, being able to run databases for $0 in licensing on small-ish, commodity servers has led us to use more and more of them, rather than giant, central, shared databases. This amplifies all of the above effects. It also has important 2nd-order effects on the social organization of our software-building endeavors. When databases were shared by many apps, it made sense to think of them as their own services. When each database is built and run to service exactly one app, that makes less sense, plus it's also easier to push everything onto the devs (as above).
Make no mistake, all these companies still need DBAs. Their schema designs are awful, their query performance is worse, who knows if their backups work, etc etc etc. But we don't see it, because there are so few people working in software today that understand what a DBA is, or ever met (and worked with) a good one.
It's rewarding work if you can get it. I still love doing it. But outside of the Oracle and Microsoft ecosystems, it's not really a career.
I worked with a dba around 14 years ago, and then again in 2016. Those were both at corporate jobs. Other than that I've worked for small businesses, start ups and as a contractor and that's all been fullstack. I think specialized roles make for better software and small teams trying to support the whole stack with their javascript bootcamp training are the biggest reason so much tech is absolute garbage these days.
So I hope you find from other people that this is still a good role to pursue and that you do become a dba and work as a specialist with highly skilled teams and make the code world a better place.
For a potentially non-indicative view: I grew up in the modern software era, whatever that means to you (frameworks, kubernetes, 500mb react web apps) and have a lot of hiring authority myself and have a lot of cross functional experience, including SRE and working in an actual data center (so loud). imo DBA is useful at the extreme edges of performance after product market fit has been found and 90% of software has been written and it’s time to optimize. I wouldn’t personally hire a DBA before that as it would be premature optimization, especially considering “modern” approaches to profiling queries. I expect my entire team of SWEs to have profiled any DB queries and found low hanging fruits, and also lean towards fully managed database services like RDS.
That being said, that’s a startup opinion. There are lots of F500s that will want them in their data centers. So like cobol programmers, I don’t think there will be a problem getting employment somewhere in the foreseeable future.
Yes. Lots of companies employ them. They tend to be less flashy companies than the kind you see talked about on here. Think banks, insurance companies, manufacturing companies, etc. Not startups / pure tech companies - while they may have them, you're more likely to encounter them mashed together with other roles (data engineer, devops, etc...) in the tech companies that tend to chase the latest dev tool fads.
Personally, I'd take a proper DBA over modern data engineers any day - there's a ton of value in knowing how to set up, maintain and use an effective SQL database. It's just not a place you find shiny new tech, so people regard it as boring and old. I'm biased though - the beginning of my career I was a DBA in a boring industry, but it taught me A TON that still has an impact on my skill set 30 years later even though I'm now working in totally different areas.
Like everything, for legacy software (of which there is a lot in the enterprise) DBAs are still needed.
I haven't seen as many DBAs in the last 15 years. In that time, two things have happened:
1. Servers have become faster (obviously). I spent a lot of time in the late 90s and early 2000s optimising databases, from the data model and query plans to physical hardware. We forget how slow hard drives were, and how challenging a database cluster is to run on 4GB of RAM. These days we don't spend much time worrying about database scalability for most cases. There was a time where many businesses were pushing the upper limits of database server performance for hardware available at the time.
2. SQL databases have become less central to the solution. Everything used to be modelled in third normal form and the SQL database was the 'single version of the truth'. Now we may put some data in SQL, some in a document database, and some on a queue to be saved in a datalake.
If you are asking specifically about SQL DBAs then yes, the career is fading. However, as others on this thread have pointed out, there is always a career for people who love data, and know how to work with it.
I hope not. We need to hire a Postgres DBA at Fivetran, we have more or less a single Postgres database with all our state, and it’s become clear that we need a full time person to optimize it.
I've not seen many 100% DBA roles except at "large" businesses (though, to be fair, I've only had 3 professional IT salary jobs in 12 years)... And it's arguable to say the people I'm thinking of are 100% DBA... The one that comes to mind best as "100% dba" was responsible for working with devs to improve queries, work with auditors, help manage deployment of certificates (not just DB related certs), and monitor DBs for unusually activity... On top of typical backups, migrations etc.
I would likely not suggest a "DBA" as a career, but it is a very valuable skill set... To get a job at anywhere but a large business... I feel you need one other hat at least.
It seems like Datadog is defining a new career title called DRE (Database Reliability Engineers), which is IMHO a next-generation of DBA, just like DevOps.
The role is called data engineer now. It's just one of many data engineering roles. Data engineers at a non-tech company could be 1 person holding up the entire system doing administrative tasks on it so to speak. It could also be one of many persons that work towards holding up Youtube's ML recommender systems.
I think just in general data engineer is a better term to find the same role across a lot of companies today.
DBAs are done. A certain level of database proficiency is demanded of all the other engineers and non-engineers. And if you are really working on a super-awesomely-complex system that needs super awesome scaling, then..... you make shitty queries anyway and just increase the size of the memory cache or the number of database server instances. What!??? Heresy! Well okay then, go find a bad query and improve it so you get a nice bonus later. See, look, still no DBA on staff.
Major products are released and operating right now with incredibly low performant queries and uses of databases. The infrastructure people and services keep the databases running.
DBAs are a subset of sysadmins, and the sysadmin role is being replaced by devops and offloading more and more functionality to cloud as-a-service products.
Even at large companies, you'll often see databases being run by "data engineering" and SRE teams rather than pure DBA teams. This isn't just a naming difference: the background and mindset expected in these roles is engineering, not system administration.
You're going to be most likely to find traditional DBA roles at companies that run a lot of legacy on-premise software: banks, defense contractors, government jobs, older large tech companies like IBM and HP, etc.
I’m a DBA at a small university. I’ve found that as we move more stuff to “the cloud” on various platforms, my job is much less performance and administration, and much more development. I spend most of my time writing data exchange jobs (between vendor systems) and supporting our reporting/analytics users.
I don’t plan to continue in administration work when I move on from my current role—partly because I’ve found the development to be more fun, and partly because I’m seeing the writing on the wall.
CTO of TiDB here.
TLDR, yes, DBA is still a good job, but to embrace new technologies.
From a database developer's point of view, I have always admired the work of DBA, although I can develop the kernel of the database, I can also write a SQL engine, never been good at writing complex and efficient SQL, totally different skills.
I have noticed a trend in the database field in recent years: Everything goes to the cloud.
Because of cloud is more and more popular, and the cloud is providing fully managed database service, it is hard to say the trend of traditional DBA job is getting less and less, the DBA expert is still extremely popular, However DBA will alive and can be more welcome, because there are more and more new technologies are evolving fast, just like what we're proudly building: TiDB, an HTAP database provides SQL interface, support complex analytic workloads, as well as the OLTP load. DBA could reuse their skills, and even better, they are not just DBA, they can also act as a data expert, help application developers to optimize SQL queries, and schema design, and help them to design and optimize systems.
A good example is my recent little side project: https://ossinsight.io/, although the architecture is simple, it contains some complex SQL in there, I turned to my DBA friends to get this application done correctly and efficiently.
This topic inspired my co-founder and I to build dbsupervisor.com as a tool to monitor database health, and offer opinionated suggestions to improve product uptime and performance.
As others have pointed out, traditional DBA roles are fading away due to cloud offerings abstracting many of the problems DBA’s would otherwise be responsible for. Hardware is cheaper and more available which means that companies throw more RAM/CPU at their problems, whilst areas like index planning, performance tuning, query optimisation, and resource monitoring take a back seat. In our experience, this eventually catches up to you in the form of poor performance and unhappy customers.
We believe that tech can augment most of the insights offered by a traditional DBA and help avoid these pains early on, especially for smaller teams with limited database expertise. We’re currently in private beta, but will be opening up more broadly in the coming weeks. We’d love to get your feedback on our approach - sign up for the beta or AMA if you have any questions!
Recently I’ve thought it might be good to go back into consulting as a Data Architect. I would happily partner up with a good product if it there was (hands waving) “synergy”...
[+] [-] forinti|3 years ago|reply
But, I now understand the problem. At my organisation we have only 2 DBAs, and we can't give all of our time to this task.
To be a DBA is an endless battle against the entropy some devs try to create. The data model should really always be validated by someone with more knowledge and experience. Also, some young devs come up with crazy ideas (for instance, we should not use foreign keys). Others want to load insane amounts of data to process in the database, instead of preprocessing, or using a staging database. There is a continuous stream of bad ideas being implemented.
And when a bad idea gets implemented, it is really hard to undo it and it tends to just create more chaos: you need materialised views to get around bad modelling, or weird views to compensate for duplicate data, etc.
So, yes, a DBA is a really important role. With regards to it being a good job, it depends if your company takes it seriously and puts you in the development process, otherwise you'll be in for a lot of stress.
[+] [-] brightball|3 years ago|reply
The older I get and end up around younger developers, I find that this has become a rare superpower. These days, if I’m consulting on a problem for somebody I can almost assume that it’s a database problem. Either needs to be tuned, indexes are missing, data has been corrupted or duplicated due to lack of constraints, excessive single writes or any number of other issues.
I never thought something that was always a required skill set to suddenly become rare and shocking.
[+] [-] giancarlostoro|3 years ago|reply
My favorite was "we shouldn't use indexes it slows down the DB" by a junior in regards to us collectively telling him to add indexes to a MondoDB collection. When we told him to research it, we meant how to do it.
[+] [-] throw0101a|3 years ago|reply
Are there resources (e.g., books) that one can get ideas about good data modelling?
[+] [-] throwaway6532|3 years ago|reply
This describes all aspects of software development.
[+] [-] hyperman1|3 years ago|reply
[+] [-] aejnsn|3 years ago|reply
[+] [-] arethuza|3 years ago|reply
Some Microsoft enterprise products were like this (e.g. Dynamics AX) - try understanding 5000 odd tables without any foreign keys!
[+] [-] MarionG|3 years ago|reply
Not using foreign key constraints is a good idea. Occasionally. Very occasionally.
[+] [-] unknown|3 years ago|reply
[deleted]
[+] [-] Twisell|3 years ago|reply
I would argue that it's way more accurate to say this role is actually outsourced in most organizations with all the tradeoffs that come with that strategy. I begged my former employer to subscribe to a good PostgreSQL support company. Guess what the one we picked is constantly looking to hire new DBA to sustain growth...
I was in database engineering/administration position for the last 10 year on a small company. It was exhausting because management wouldn't care to rely one guy to manage the whole data stack. It was rewarding because the CEO saw me as probably smarter than I was because I knew SQL well enough to produce outstanding value as a single individual.
Often reading HackerNews I felt like an imposter because NoSQL was the new hype and database was seen as a thing of the past... well trends reverse... and people will always need reliable systems to handle business critical data.
So maybe in a few year after few major Cloud data horror story DBA will become the new trend. No one know.
But learning how to handle data? That will always be handy no matter the underlying technology.
[+] [-] citrin_ru|3 years ago|reply
Or in-sourced to developers. In places I've worked it was a developer's responsibility to ensure that a DB is used properly. None of them knew the DB good enough (they need to write code and have no time to read DB docs) so the result was mixed at best.
[+] [-] hrpnk|3 years ago|reply
Next, the DB experts help improve efficiency. It's too easy for engineers to over-provision a DB instead of rewriting the query or adding an index or two. Lastly, engineers flock to new datastores on the Cloud that may not be the best fit for the use cases at hand. So advising teams on which datastore to pick is another angle of the role.
[+] [-] biztos|3 years ago|reply
If we had had anyone who was actually good at that stuff, besides the few top people who were needed elsewhere, that person would have emerged a hero.
[+] [-] krageon|3 years ago|reply
[+] [-] sascha_sl|3 years ago|reply
It's also a role that's thinning out, because these managed offers are good enough a lot of the time and developers are using the most common parts of the feature set on offer.
[+] [-] Cthulhu_|3 years ago|reply
[+] [-] datalopers|3 years ago|reply
[+] [-] jve|3 years ago|reply
Remember that HN is a bubble and world is a much larger place :) No, it is not disappearing "quickly".
[+] [-] gigatexal|3 years ago|reply
[+] [-] gadders|3 years ago|reply
- the "Dev DBA" who makes sure ER diagrams are constructed sensibly, knows how to look at an explain plan to see where queries are going wrong, maybe even the odd stored proc (heresy!) - the "Sys Admin DBA" who makes sure the database has enough space, rollback segments are big enough etc etc.
[+] [-] DeathArrow|3 years ago|reply
[+] [-] NikolaNovak|3 years ago|reply
It is astonishing how much a poorly tuned data request / SQL, or poorly designed data model / table, can impact performance by multiple orders of magnitude. It is also fascinating how much addition of specific index or tuning statistics can help execution. There comes a scale where throwing more hardware on performance problem is prohibitive over hiring a good DBA, as much as current thinking is that "hardware is cheap". But on a regular basis I'm seeing experienced, serious, dedicated developers create SQL that is functional correct and reads 10 billion rows into buffer in order to spit out a single row answer (sometimes of course that's necessary, but usually not :)
[+] [-] that_guy_iain|3 years ago|reply
Even when it's clear as day that the data model is incorrect they often just ignore it. They blame MySQL for not scaling while having ten joins per query and using ORs like they're going out of fashion. Say it can't be done in another way, even tho clearly it could.
Or say how the benchmarks for Elasticsearch are specicially crafted, you can say that when you get 30k per second and they get 50k, but when you get 50 per second and they get 50k per second it's not their benchmark that is specially crafted it's your data model is crap.
I honestly think the thing that costs most companies the most in their hosting is developers not using databases correctly and spiking the hosting costs. I literally saw one company have to spend 20,000 a month for 500 requests per second on elasticsearch.
[+] [-] techdragon|3 years ago|reply
Adding a DBA to a team is like adding an opinionated senior developer, sometimes you need that experience and sometimes it’s just going to be a total fucking pain in your ass.
If you feel you need better data modelling skills in your developers, don’t hire a DBA, hire developers that know how to do data modelling.
[+] [-] BenoitEssiambre|3 years ago|reply
[+] [-] nop_slide|3 years ago|reply
[+] [-] arsome|3 years ago|reply
[+] [-] donatj|3 years ago|reply
He got let go for basically having a really bad temper, and we decided not to look for a new DBA. I was sad he was gone but also glad to not be getting screamed at. It feels silly, but I've got really unresolved and mixed feelings about the whole thing. I genuinely really liked the guy, loved talking music and movies with him, but he would heel turn in an instant and just ruin your day.
Honestly wish we had a DBA, I ended up picking up a lot of his responsibilities. I'd love to have someone I could learn from. Basically, I'm getting real sick of optimizing reports.
[+] [-] slotrans|3 years ago|reply
It's important to understand why it existed in the first place. A long time ago, if you were building business software on a database at all, it was a very costly commercial database. Imagine 6 figures a year for the software licenses, the same amount again for a support contract, and a similar amount for the hardware to run it on. If a company is spending $1M+ a year on a critical piece of software infrastructure, that is very complicated in addition to being very expensive, they will usually be very open to spending more on top of that in salary to employ experts to look after it. Those experts are your DBAs.
A funny thing happened when MySQL and Postgres made databases "free": companies (irrationally, IMHO) stopped being willing to pay for experts to run their database software. That duty became a secondary responsibility of developers or operators or both, none of whom really wanted to do it or cared to learn how to do it well. There are counterexamples, sure. Some companies get big enough, and lean on their databases hard enough, that they cultivate true competence in them, but it's rare.
And then along comes AWS RDS (and similar), automating away the most tangible aspects of what DBAs did: HA, backup/restore, configuration, provisioning. There's still plenty left for DBAs to do, but it's stuff people don't understand the value of. As above, the residual DBA duties silently fall to developers and operators.
Also, being able to run databases for $0 in licensing on small-ish, commodity servers has led us to use more and more of them, rather than giant, central, shared databases. This amplifies all of the above effects. It also has important 2nd-order effects on the social organization of our software-building endeavors. When databases were shared by many apps, it made sense to think of them as their own services. When each database is built and run to service exactly one app, that makes less sense, plus it's also easier to push everything onto the devs (as above).
Make no mistake, all these companies still need DBAs. Their schema designs are awful, their query performance is worse, who knows if their backups work, etc etc etc. But we don't see it, because there are so few people working in software today that understand what a DBA is, or ever met (and worked with) a good one.
It's rewarding work if you can get it. I still love doing it. But outside of the Oracle and Microsoft ecosystems, it's not really a career.
[+] [-] w4ffl35|3 years ago|reply
So I hope you find from other people that this is still a good role to pursue and that you do become a dba and work as a specialist with highly skilled teams and make the code world a better place.
[+] [-] ohyoutravel|3 years ago|reply
That being said, that’s a startup opinion. There are lots of F500s that will want them in their data centers. So like cobol programmers, I don’t think there will be a problem getting employment somewhere in the foreseeable future.
[+] [-] porcoda|3 years ago|reply
Personally, I'd take a proper DBA over modern data engineers any day - there's a ton of value in knowing how to set up, maintain and use an effective SQL database. It's just not a place you find shiny new tech, so people regard it as boring and old. I'm biased though - the beginning of my career I was a DBA in a boring industry, but it taught me A TON that still has an impact on my skill set 30 years later even though I'm now working in totally different areas.
[+] [-] Delphiza|3 years ago|reply
I haven't seen as many DBAs in the last 15 years. In that time, two things have happened:
1. Servers have become faster (obviously). I spent a lot of time in the late 90s and early 2000s optimising databases, from the data model and query plans to physical hardware. We forget how slow hard drives were, and how challenging a database cluster is to run on 4GB of RAM. These days we don't spend much time worrying about database scalability for most cases. There was a time where many businesses were pushing the upper limits of database server performance for hardware available at the time.
2. SQL databases have become less central to the solution. Everything used to be modelled in third normal form and the SQL database was the 'single version of the truth'. Now we may put some data in SQL, some in a document database, and some on a queue to be saved in a datalake.
If you are asking specifically about SQL DBAs then yes, the career is fading. However, as others on this thread have pointed out, there is always a career for people who love data, and know how to work with it.
[+] [-] DeathArrow|3 years ago|reply
What's that career name? What would be a path towards that career?
[+] [-] georgewfraser|3 years ago|reply
[+] [-] ldoughty|3 years ago|reply
I would likely not suggest a "DBA" as a career, but it is a very valuable skill set... To get a job at anywhere but a large business... I feel you need one other hat at least.
[+] [-] vira28|3 years ago|reply
Drop me a line: vignesh@ or reply here.
[+] [-] dragonly|3 years ago|reply
https://www.datadoghq.com/careers/detail/?gh_jid=1825950
[+] [-] data4lyfe|3 years ago|reply
I think just in general data engineer is a better term to find the same role across a lot of companies today.
[+] [-] 29athrowaway|3 years ago|reply
Database administrators keep databases running, managing access and security, doing maintenance tasks, running migrations, etc.
Data engineers are about maximizing the value of existing data by transforming it, aggregating it, etc.
[+] [-] vmception|3 years ago|reply
Major products are released and operating right now with incredibly low performant queries and uses of databases. The infrastructure people and services keep the databases running.
[+] [-] twblalock|3 years ago|reply
Even at large companies, you'll often see databases being run by "data engineering" and SRE teams rather than pure DBA teams. This isn't just a naming difference: the background and mindset expected in these roles is engineering, not system administration.
You're going to be most likely to find traditional DBA roles at companies that run a lot of legacy on-premise software: banks, defense contractors, government jobs, older large tech companies like IBM and HP, etc.
[+] [-] escapedmoose|3 years ago|reply
I don’t plan to continue in administration work when I move on from my current role—partly because I’ve found the development to be more fun, and partly because I’m seeing the writing on the wall.
[+] [-] c4pt0r|3 years ago|reply
From a database developer's point of view, I have always admired the work of DBA, although I can develop the kernel of the database, I can also write a SQL engine, never been good at writing complex and efficient SQL, totally different skills.
I have noticed a trend in the database field in recent years: Everything goes to the cloud.
Because of cloud is more and more popular, and the cloud is providing fully managed database service, it is hard to say the trend of traditional DBA job is getting less and less, the DBA expert is still extremely popular, However DBA will alive and can be more welcome, because there are more and more new technologies are evolving fast, just like what we're proudly building: TiDB, an HTAP database provides SQL interface, support complex analytic workloads, as well as the OLTP load. DBA could reuse their skills, and even better, they are not just DBA, they can also act as a data expert, help application developers to optimize SQL queries, and schema design, and help them to design and optimize systems.
A good example is my recent little side project: https://ossinsight.io/, although the architecture is simple, it contains some complex SQL in there, I turned to my DBA friends to get this application done correctly and efficiently.
[+] [-] mrpadie|3 years ago|reply
As others have pointed out, traditional DBA roles are fading away due to cloud offerings abstracting many of the problems DBA’s would otherwise be responsible for. Hardware is cheaper and more available which means that companies throw more RAM/CPU at their problems, whilst areas like index planning, performance tuning, query optimisation, and resource monitoring take a back seat. In our experience, this eventually catches up to you in the form of poor performance and unhappy customers.
We believe that tech can augment most of the insights offered by a traditional DBA and help avoid these pains early on, especially for smaller teams with limited database expertise. We’re currently in private beta, but will be opening up more broadly in the coming weeks. We’d love to get your feedback on our approach - sign up for the beta or AMA if you have any questions!
[+] [-] no-s|3 years ago|reply