Ask HN: Does anyone actually use data structures/algorithms at work?

[rbnsl]

Sitting at around 4 years of experience now and last month was my first time actually implementing a graph traversal in prod, and then last week a skip list. It was fun, and I loved being able to instantly link the problems I was facing to algo/structures I knew would work, but it seems it happens so infrequently to not justify the emphasis we place on these in interviews.

I imagine any regular SWE at <insert-product-focused-company> just ends up using libraries that themselves implement all the data structure/algo pieces, leaving you with just abstractions.

Anyone here regularly using/thinking about these? And I don’t just mean small things like when to use an array vs a map but the more complex stuff

[cmpalmer52]

About 20 years into my career, I wrote a tracing system for three phase electrical utility networks to help manage and predict outages and their impact. Basically, each phase was a dendritic n-tree with the phases superimposed on each other because of the way the transmission lines were drawn in GIS (where a three phase line with ground was usually represented as a single line feature with attributes designating which phases were present).

So my solution was to turn each phase’s graph into a binary tree by converting the n-tree to a b-tree where each node has a parent, a child, and a sibling to make traversal easy. Then I gathered all of the nodes into a single ordered array of structs and used array indices to reference parent, child, sibling for each phase.

To trace up, you selecting the starting node and the phase and it would walk up the graph to a source or you could specify a source node and transverse down by phases to create a downstream trace showing all affected customers.

Each node also had protective devices, like fuses and switches, that could be activated/deactivated or predicted as sources of outages.

At the time, I could downstream trace to every customer of a substation in a second or less when most GIS based network traces took 10-15 seconds sometimes. Upstream traces were almost instantaneous. Plus, our customers often converted CAD to GIS and the lines lacked spatial connectivity without a lot of cleanup, but my trace engine could handle disconnected lines and devices that weren’t spatially connected by using attributes.

It was the biggest use of my data structures and algorithms knowledge that I’ve used and it was good enough for a software patent.

[BOOSTERHIDROGEN]

What would be the terms of this software ? e.g xz for extraction tools

[gavin_]

I think every engineer needs a basic understanding of data structures and algorithms to be successful in problem solving during their day to day. Algorithms teach out how to think about problems and Data Structures give you the tools to solve them.

Whether or not you use them in your work depends on the industry you work in. I reckon at Databricks or <insert-tech-focused-company> they used plenty of algorithmic knowledge and lots of complex data structures early on. But for <insert-product-focused-company> like you mentioned, I think practical product development skills are way more important -- architecting BE systems and knowing how to organize FE components in a modular and extensible way. Thinking like a user becomes more important than thinking like a scientist.

[BOOSTERHIDROGEN]

Any intro books about this ?

[oxfordmale]

I use them everyday. They are built into the standard Python libraries and SQL databases I use :-)

Unless you work at the cutting edge, developing new database engines or cutting edge algorithms, you rare use data algorithms. My day to day challenges are not the algorithms but understanding requirements from humans.

[JohnFen]

We do where I work, but we don't make consumer products.

We need efficiency and control, and that usually requires developing our own data structures and algorithms rather than using what's in standard libraries. This is amplified by our need to keep the use of third-party libraries to an absolute minimum.

[NoddyCode]

I don't really use them directly, but knowing about them helps me write more performant code. I know to ask questions like "which of the available sort functions is best for data that is mostly sorted" and "what's the best way to navigate this deeply nested object?" They also help you recognize certain patterns and apply known solutions before trying to reinvent the wheel. SO many problems end up being graph problems that have been solved time and time again (and sometimes you can recognize a Traveling Salesman problem before you start ripping your hair out).

Plus, writing algorithms gives you lots of practice with recursion and iteration, and knowing when to use which where. And it's fun! You may not use the knowledge directly, but you do pick up a lot of little useful nuggets for real enterprise development.

[jbjbjbjb]

The more complex stuff like graph traversal, topological sort has come up in just a couple of scenarios with scheduling. I actually had to implement my own topological sort one time. Caching and memoization is helpful and that’s more frequent, cache invalidation is hard. I used a queue last week and that always feels good! It’s certainly not a day to day thing.

I think a lot of developers, even ones that studied computer science, don’t see it as useful or relevant. I feel like that makes them underestimate what the modern computer is capable of and also want to reach to the next shiny object to add to the stack.

[PaulHoule]

This week I've been working on a totally run-of-the-mill React app and thinking a lot about little algorithms and data structures things like what's the best way to form the data required to populate a dropdown by merging three different data structures. I am mainly using built in data structures like arrays, sets, maps, objects, etc. but there is plenty of art into how to put them together into code that runs fast, is easy to maintain, etc.

[usgroup]

I do, sometimes. It comes with trying to optimise things. If you want to make "it" go fast or fit into RAM or whatever, at some point you may have to consider structure deeply. Take the "1 billion row challenge" as a good example of one way data structure considerations naturally emerge.

[qwertyuiop_]

Most of the algorithms have already been abstracted away with hundreds of thousands of libraries and frameworks, so developers don’t even have to think about algorithms unless its a specialized case.

[giantg2]

Nope. Like you said, it's basically informal and all abstracted.