Ask HN: Soliciting critique/feedback for your technical architecture?

[fizwhiz]

Lately I've been interested in pursuing a whole range of side projects that bear no resemblance to my day job (vanilla java Spring mid-tier service in a large enterprise company). While I know experiential learning is best by building, making mistakes, incrementally improving (rinse, repeat) I'm curious to understand how folks here get their projects' technical architecture critiqued. Please note that I'm not dismissing the value of books/blogs/talks but I'm talking about once all those materials have been surveyed fairly well, what other options are available to validate your thought process.

Do you post a high level overview online and get the community to provide feedback, or do you rely on friends/co-workers to chime in and provide some insight? The kinds of questions I'd like to get answers to are around something to the effect of "Hmm, your data pipeline should use Kafka and elastic search for XYZ reason instead of RabbitMQ and cassandra for the scale and access patterns you're interested in." "For the fault tolerance you're interested in, erlang would be a good fit" "Here's some other projects/case studies that made ABC tradeoffs that are worth surveying..." Trouble is most of the people I'm surrounded by at work haven't expanded their skills beyond what's expected at the job, so I don't find discussions with them about the architectural possibilities as illuminating as I'd hope.

Are there any reliable ways to get answers to such questions from people who are experienced? Solving different flavors of problems seems crucial to truly level up as an engineer, and I just want to make sure I can get feedback so that I can immerse myself intelligently.

[contingencies]

Rarely you will find one gleaming Solution - even if you are capable of identifying and discussing all important factors prior to the design, and know perfectly all of the properties of various pre-built systems that may be used to assemble it. Almost always it is necessary to proceed with an architecture that has been segmented in to manageable components and APIs, knowing that you can swap out individual portions as issues are identified. That said, personally I really enjoy this kind of thing and have actually been considering starting a consulting business in the same vein; I would be willing to have a look for you just to get a better understanding of your what you are looking for and gain some insight in to potential customer requirements in this area should I decide to proceed with a consulting practice.

[bradleyland]

> Are there any reliable ways to get answers to such questions from people who are experienced?

In my experience, the challenge with soliciting this kind of feedback is in filtering the responses. Opinions are formed for a variety of reasons. Some are well formed from experience, while others are simply dogma:

Authority Figure X says to use Solution Y.

or cargo cult:

Successful Company X uses Solution Y, so you should too.

It's worth considering why authority figures recommend specific solutions, but that informs you as to the reasons why you should use a particular solution, not that you should always use a particular solution.

Giving a good recommendation first requires that the offerer of the opinion have the qualifications necessary to be considered an authority on the problem domain; second, that they invest the requisite time in understanding your unique set of circumstances; and finally, that they are prepared to set aside any dogmatic beliefs they have developed along the way. You must be prepared to filter the opinions you receive in order to avoid incorporating advice affected by these shortcomings.

Rather that solicit advice on your entire infrastructure, you should do two things:

1) You should seek to understand the technologies you're implementing to a depth that allows you to identify potential pitfalls. In other words, understand what a particular solution is good and bad at.

2) Try to hire/work with people who specialize in specific problem domains. A jack-of-all-trades can only have so much depth.

[chris_va]

(This doesn't really answer your question, but hopefully is still useful)

I actually do this for startups, but it's a bit of a specialized and difficult art. Answering specific questions about technology stacks requires having used them before, and most people only have experience with a small subset of them (myself included, so I probably couldn't answer half of your questions). As a result, I often break down the question into phases.

First, broader questions on design are actually a lot easier to answer. For example: "for problem X, should I use a producer/consumer model + sharded key/value or a direct RPC to a search stack?". Or even more open ended: "on this budget, how can I consume the twitter firehose in realtime to do X?".

Second, the more specific "should I use RabbitMQ or Kafka" for my producer/consumer component requires specialized knowledge. Once you break your question down to that level of granularity, though, it is relatively easy to find an expert.

[wumbernang]

Realistically everything is mostly shitty architecture and nearly on fire all the time.

The best critique is surviving production and constant iteration and tackling limits and problems as they become apparent.

If you see a problem, research solutions and hit the cycle again. Sometimes things don't work.

There's not much science in it I'm afraid.

[jobjh797878]

I like to start with a discussion, rather than a presentation and Q&A. For me, I find that really understanding the problem is the hard part. Once I really know what I need, what my requirements are, the options fall into place.

Today, for example, I'm going to have a whiteboarding session with one of my employees. We're going to whiteboard the problem we're solving, again, and again, until we both can describe high and low level details in our sleep.

Then we'll do a few searches and leverage other people's experiences to pick our technologies.

The also works for pure architectural discussions. You have a queue between your web server and web workers -- why? etc.

[kisna72]

I have had need for critique on my architecture. What helps me is, I go to a tech meetup, and talk to a bunch of people. When I am working on a new side project, I will go and tell people about it and show them how I am planning on implementing it to solicit their opinions. I sometimes will ask a few co-workers to go to lunch with me with a purpose to talk about the project. Luckily I have at least one co-worker who loves advising and critiquing on my design decisions. Depending on where you live, I would recommend to join a tech meetup.

[iamlolz]

I've been wondering the same thing lately. I tried posting to the NodeJS subreddit, but it's near deserted.

[fizwhiz]

Yeah, I would imagine questions like these that stretch beyond a single runtime or language/framework require experienced people to engage in a meaningful discussion which is why sometimes the topics don't get as much love because it's hard to swiftly do justice to them online.

[hoodoof]

I think there's a StackExchange for that isn't there?

Try posting it to reddit/r/programming too

[fizwhiz]

AFAIK, Stackoverflow gets used for these types of questions a fair amount in a very ad-hoc manner (YMMV in terms of successful responses) and there is a "Code Review" stackexchange; I haven't yet stumbled upon a strictly dedicated forum which revolves around critiquing "system architecture".