[bbfrhd]

We've discussed that in the past and will likely discuss again in the future.

The value of the challenges over Github reviews are 1) allows us to anonymize the process, 2) eases level-setting and adjustment across the reviewers to prevent a "hard grader" throwing off the overall review, and 3) doesn't bias against those whose workplaces did not allow or encourage open source contributions. It gives us a consistent baseline as a starting point.

GitHub (or other open source code contributions) can come into play after the initial code challenge portion is passed and can be a great point of discussion in the interviews. We're definitely advocates for open source, especially in terms of making our work on behalf the people of America available to them in source format as much as possible.

For us, at least, we've found having a consistent first step to the recruiting process to be very beneficial in terms of giving everyone an equal starting point for consideration. It's proven difficult to come up with a method for weighing side projects versus open sourced work projects versus contributions to existing projects in a manner that has felt consistent and fair to candidates.

[navalsaini]

BRIEF: The code round cannot really be a differentiator. It should be used as an elimination round and carry grades to next rounds.

REASON: Some people might want to do a bare minimum (because they are busy) and other who have more space can go overboard. It might be normalizing at your end, but you are ignoring that all test takers will not treat the test alike (its not a SAT for them).

DETAILS:

hmm what will a 4-8 hour coding challenge tell you that an open source project wont? If you give someone a coding challenge, possibly these are the things that you would rate the submission on. An open source project can equally show below.

1. Codeing test - Basic solution

  a. Bare bone working application
  b. Breaking complexities into testable parts
  c. Usage documentation or a read me file
  d. Commenting in code

Bonus points: for using any interesting algo

2. Architecture test - Improves over the basic solution (faster, more maintainable, etc)

  a. Importing data
  b. Database design
  c. API design
  d. Adding testcases

3. Product thinking - Improves over the problem statement

  a. Building on top of the scope 

4. Misc. (... stretching it a bit more)

  a. Did I learn something new doing it? - tech wise, product wise, etc
  b. How much time do I allocate for it?
  c. When do I allocate time for it?
  d. Is it similar to any older project, etc...?

((I had these listed somewhere in the mail))

[bbfrhd]

Yes, I meant to agree with you that we can learn the same things from open source as a coding challenge, which is why we've considered it.

We're still a small company (but growing quickly). The code challenges are a simpler way to get the desired end goal. As we grow larger and have more capacity to devote to designing and refining an additional process around open source reviews, I think it's something we may explore. I'm not a decision-maker on that, so just my opinion. No disagreement in principle on this point, we just made a practical decision based on our constraints at the moment.