top | item 20589102

Big O Notation – Using not-boring math to measure code’s efficiency

178 points| rspivak | 6 years ago |interviewcake.com | reply

95 comments

order
[+] abalone|6 years ago|reply
I've noticed there are SO many of these "coding interview prep" courses lately. Like, obviously it's a hot job market, but there's just so many and of a certain vibe that it seems like some are selling a dream.

There's a common narrative too, that it's all about algorithm question prep and it's a "game" you can win. One youtuber who recently landed a ~$275K TC gig said it's not about being "intelligent" it's about practice and rapid pattern recognition (he also was pitching his own coding interview prep course). Said he forgot half of it after getting the job.

This could all be true but this narrative is coming from sources that have a financial interest in making you believe that. What do you think HN? I mean I know algorithm prep is an important part of the interview but.. there's just something sketch about all these outfits I can't quite put my finger on.

(I also wonder if the people who get hired based on prep vs. aptitude then have a kind of survivor bias and hire others who are strong on this limited range of pattern recognition... could it be a problem for the industry?)

[+] dntrkv|6 years ago|reply
It depends on the company, but most of the FAANG company (and others that try to imitate them) interviews come down to how quickly you can get the optimal solution on the board. It has very little to do with how well you can code day to day.

Most interviews have 4 sessions broken up in the following manner:

1 hour: coding

1 hour: coding

1 hour: lunch

1 hour: system design

1 hour: cross-functional

In my experience though, the first two are the ones that really make or break candidates, if you don't pass both, you're not getting hired. And yes, most of those interviews just test your knowledge of how quickly you can figure out the right data structure and algo for the question at hand. And even if you know all of the data structures and the applications of each, it can be incredibly difficult to solve the problems in the allotted time.

The other funny thing is that your previous experience doesn't do anything for you other than maybe getting to skip the technical phone screen. Everyone goes through the same process. e.g. https://twitter.com/mxcl/status/608682016205344768?lang=en

On the bright side, it does look like many companies are now realizing this isn't a great way to evaluate someone's ability and are beginning to incorporate more practical exercises.

[+] save_ferris|6 years ago|reply
A major force driving this is the employers who choose to conduct interviews in this way.

I'm a developer with about 6 years experience in full stack web app development (Rails & Django mostly), and I recently got my ass absolutely kicked during a couple of live coding challenges that were very heavy on theory (also worth mentioning that I'm a self-taught dev without a CS degree).

I started talking to some former coworkers about my experience, and I got 2 main responses: A) it's imperative to have a CS foundation regardless of your degree if you want to advance in your career, and B) employers only consider prior experience to certain degree.

The two interviews that I bombed involved heavily contrived coding challenges that tested CS fundamentals in ways that I've never encountered in a professional environment, so it doesn't surprise me at all to see services like this popping up.

[+] ma2rten|6 years ago|reply
I work at Google. I think it's half true.

I would not have passed the Google interview if I had not prepared. After 3 years, I have completely forgotten most of the prep. I couldn't even implement a binary search right now.

That said, I do have an innate ability for problem solving. I know another person, who has prepared significantly more than me (multiple months full-time) but still wasn't able to get an on-site interview. In my opinion, it requires at least a minimum level of intelligence, luck and preparation.

[+] hello_moto|6 years ago|reply
> I've noticed there are SO many of these "coding interview prep" courses lately. Like, obviously it's a hot job market, but there's just so many and of a certain vibe that it seems like some are selling a dream.

It's akin to the Fitness world. There are tons of people selling proteins whey, their fitness apps, fitness set "reps", etc. Can't blame them, they're selling what people want.

[+] weq|6 years ago|reply
I have found pair-programming with a dev on a current problem you have in your codebase to be the best litmus test.

If u want to give "quiz" interviews, u will get employee's good doing them. Like those CS grads in my class who copied eachothers take-home assignments yet still passed because they were good at memoizing.

Furthermore, this is the first training session for your future employee - you are inducting them into your companies culture. If "answering meaningless questions for the sake of it, is what you want to portray to them. So be it." They have been warned!

Selecting for a certain subset of people, is a business decision. Be aware of what interview practices mean in the context of how that company operates. If the job add has a sticker list of tech-hype, a promise of re-dev'ing a monolith into k8 microservices, with a leetcode interview, you already know alot about that company!

[+] dgzl|6 years ago|reply
I love your insight, but I don't know much about these courses. Even though I am myself prepping for an interview, I'm just reading CtCi and building the structures and algorithms myself.
[+] xenihn|6 years ago|reply
I'm pretty sure that Interview Cake is at least five years old at this point.
[+] curiousgal|6 years ago|reply
> not-boring math

What's next? Gradient decent with no gradients? Fourrier transform with no functions?

If math is do boring just skip the whole thing, don't try to kid yourself by saying you're not doing it when you are in fact doing it.

/rant

[+] jrumbut|6 years ago|reply
I was really hoping that this was going to be some lively new way of thinking about or teaching big O notation, "Eulerian path runtime grows like the velocity of a car that drove off a cliff but Hamiltonian path runtime grows like the number of plague bacteria in a petry dish."

I was disappointed.

[+] thiht|6 years ago|reply
What's with this gatekeeping attitude?

Big-O notation is a useful tool in a programmer's belt, knowing the math behind it in details is less useful. What's the problem? It's the same thing as not needing to know how an engine works to be able to use a car

[+] ajmarcic|6 years ago|reply
This article misses an actual definition of Big O.

This forces us to skip crucial points like why O(n^2) truly identically equals O(n^2/2 + 5n). The author instead seems to think they are approximately the same because the lower order terms are small compared to the highest one.

We only need to explain the definition and the reader could understand why an O(n) algorithm is in O(n^2), why some O(n^3) algorithms are much faster than others in O(n^~2.4), etc.

[+] devnulloverflow|6 years ago|reply
> The author instead seems to think they are approximately the same because the lower order terms are small compared to the highest one.

In fairness, this is actually why we care in practice about asymptotic issues. The fact that there is a rigorous mathematical viewpoint that makes them strictly comparable is helpful for being confident about the non-trivial results -- but the reason we care about it is because we want to know roughly how expensive our code is (or will be).

[+] falcor84|6 years ago|reply
> As n gets really big, adding 100 or dividing by 2 has a decreasingly significant effect.

I was annoyed to see that comment about division, as of course dividing n by 2 has exactly the same effect regardless of the size of n. This is not the reason we ignore multiplicative constants in complexity analysis.

It reminded me of the joke that Earth's circumference is pretty much the same as its diameter, since the the size of pi is negligible at that scale.

[+] concert-gilled|6 years ago|reply
> This is not the reason we ignore multiplicative constants in complexity analysis.

What is the reason?

[+] blattimwind|6 years ago|reply
Except Big O Notation is exactly not measuring code's efficiency.
[+] adrianN|6 years ago|reply
That depends on your definition of efficiency.
[+] Blackthorn|6 years ago|reply
You're an engineer, aren't you? Just do the damn "boring" math.
[+] raxxorrax|6 years ago|reply
> Big O notation is [...] about how long an algorithm takes to run.

My former prof would have me spanked for an imprecise statement like this.

[+] gameguy43|6 years ago|reply
Original author here. Thanks for the post! Would stick around to answer questions but I'm on vacation in Japan right now!

We're doing a poor job with navigation on the site ATM, so here are some "you might also like" hits for you:

A piece where we derive most of the main data structures step by step, starting with naked bits in RAM: - https://www.interviewcake.com/data-structures-and-algorithms...

A reference / cheat sheet for the main data structures: - https://www.interviewcake.com/data-structures-reference

A reference / cheat sheet for sorting algorithms: - https://www.interviewcake.com/sorting-algorithm-cheat-sheet

[+] kgilpin|6 years ago|reply
It's true that most engineers are not going to implement a binary search algorithm or an optimizing compiler during the course of their employment in a typical year.

However, CS fundamentals are the language of software engineering. It's important for engineers to be able to communicate efficiently about things like runtime analysis, indexes, concurrency models, type systems, etc.

Your mechanic knows how a carburetor works, (s)he knows how brakes work. Mechanics have a common set of fundamentals that they all know and share, and can use to communicate with each other about their work. The purpose of this knowledge is not to be able to fabricate new car parts, although given time and the right equipment, it could probably be done.

The purpose of CS fundamentals is not to so much to implement complex algorithms, as to understand how to use tools and libraries that are based on them.

[+] iandanforth|6 years ago|reply
Question: At what point in a career do you expect someone to have gone from "I know Big O" to "I know how this data structure is laid out in memory on this OS?"

My experience says really good engineers know the latter, but I'd like to hear if this holds for others.

[+] bauerd|6 years ago|reply
How data structures are laid out in memory has nothing to do with the operating system
[+] MichaelBurge|6 years ago|reply

    def print_first_item(items):
        print items[0]

    This function runs in O(1) time (or "constant time") relative to its input. 
If this were C where the only types are fixed-size, that might be true. But in Python it seems like items[0] could display as 10 gigabytes of JSON.

And if stdout is piped to a process that isn't consuming its input, it might never terminate naturally.

Also, since Python has arbitrarily large integers, an expression like "index += 1" probably runs in O(log(index)) time, not O(1) time.

[+] dpzmick|6 years ago|reply
in C, it items[0] could segfault, the fault could be caught by a signal handler which does some unbounded amount of work, then populates the appropriate memory location and returns...

More realistically, the page of memory that items[0] is sitting on could be swapped to disk, requiring the OS to do some big operation.

trying to write a piece of code with the big-O you are looking for generally has a huge number of caveats. I'm undecided on whether this is a feature (abstraction) or a flaw (easily misleading)

[+] mgradowski|6 years ago|reply
Similarly, if you were to write a O(n) Python program displaying the Fibbonacci sequence, it would not run in O(n) time but it would still take O(n) operations. It only makes sense to treat time complexity literally, if operations take fixed amount of time (int addition in Python doesn't).
[+] sireat|6 years ago|reply
Am I the only one who signed up for interviewcake on a whim (similar to those January fitness club resolutions)?

Then I sort of had an epiphany. When will I need this? (or rather "I am too old for this...") I must be the perfect customer for interview cake: paid but do not use it.

I make my own work and those kind of problems just do not come up.

I mean CLRS was fun a long time ago, but it seems to degenerate into skeet shooting pretty quickly:

“Shooting skeet eight hours a month was excellent training for them. It trained them to shoot skeet.”

[+] smudgymcscmudge|6 years ago|reply
I have a question for those with a better understanding of such things. Is it difficult to calculate time or memory complexity of a function through static analysis?

It seems kind of basic, but I still find myself manually checking code instead of relying on a tool.

[+] RodgerTheGreat|6 years ago|reply
For a completely arbitrary program, it is not possible to calculate a tight bound on the time or memory complexity of that that program via static analysis. See Rice's Theorem[0].

It is possible to determine interesting properties of programs if you weaken your requirements to approximations and/or add significant constraints. You can maintain properties by construction if you only allow yourself to use certain kinds of building blocks in assembling programs. The less powerful your computational model, the easier it is to prove things about it.

The core idea of a type system is to form a symbolic representation of the semantics of your program which is simpler than the program itself, so that you can prove properties of that simpler system instead of the intractable problem of doing so for the actual program. (If you aren't careful, your type system could end up sufficiently powerful that it is itself undecidable, and you're back where you started.)

[0] https://en.wikipedia.org/wiki/Rice's_theorem

[+] Const-me|6 years ago|reply
If you need precision good enough for practical applications, it’s very hard to do. All modern CPUs are pipelined, time they spent on every instruction depends on neighbor instructions + data dependencies between them. All modern CPUs have multi-level caches, static analyzers must also simulate what data is in which levels of caches; for many practical problems RAM access dominates computations and L1 cache is ~2 orders of magnitude faster than main RAM. All modern CPUs have very sophisticated power management, they scale frequency based on temperature, they selectively turn on/off their blocks, this too affects time. All modern CPUs have multiple cores, L3 cache is often shared across them, all caches suffer heavy penalty when a line is shared across cores. And so on.

People don’t do it not only because it’s hard, also because running the function and measuring time + memory is much easier, also much more precise.

Even if you don’t care about time and memory complexity and only care about these useless O(N), it’s still very hard because complexity depends on input data. E.g. bubble sort is O(N^2) but it can also be O(N) if the data is mostly sorted already and only a single element needs to be moved.

[+] kevinventullo|6 years ago|reply
The resolution of the Halting Problem shows you can't even determine whether the complexity is finite!
[+] SamReidHughes|6 years ago|reply
Yes and no - if the function operates on a balanced tree, you might need some way of encoding the constraints the tree follows, and even just describing what “n” is. But if you’re doing merge sort, the function may take n as an integer, and recurse with n/2 and n-n/2, and that is tractable. But generally, no.
[+] zuza|6 years ago|reply
I would argue it isn't tractable.

f(n) = f(n-1) + f(n-1) has runtime O(2^n); f(n) = f(n-1) + f(n-2) has runtmie O(~1.62^n); f(n) = f(n/2) + f(n/2) has O(n log n); for i = 1 to f(n) has runtime f(n).. good luck determining a function output through static analysis

[+] pitaj|6 years ago|reply
Given a sufficiently restricted syntax, any static analysis is possible. So the question isn't especially useful with the current wording.