adamb's comments

adamb | 6 years ago | on: Tests that sometimes fail

If anyone is looking for ideas for how to build tooling that fights flaky tests, I consolidated a number of lessons into a tool I open sourced a while ago.

https://github.com/ajbouh/qa

It will do things like separate out different kinds of test failures (by error message and stacktrace) and then measure their individual rates of incidence.

You can also ask it to reproduce a specific failure in a tight loop and once it succeeds it will drop you into a debugger session so you can explore what's going on.

There are demo videos in the project highlighting these techniques. Here's one: https://asciinema.org/a/dhdetw07drgyz78yr66bm57va

adamb | 7 years ago | on: Vehicle-to-Vehicle Communication Could Replace Traffic Lights, Shorten Commutes

I think there is something romantic about the siren's song that is "we don't need object detection or velocity estimation if all objects self-report their location, velocity, and intent".

I hope you're right and that once we establish best case utility, as a community we refocus on handling component failures more gracefully.

Although it's unclear if the second and third order system effects in our financial system will ever get that sort of treatment. So let's hope driving gets a bit closer to flying (and farther from wall street) in terms of attitude towards safety.

EDIT: clarity

adamb | 7 years ago | on: Vehicle-to-Vehicle Communication Could Replace Traffic Lights, Shorten Commutes

Fair enough. Though my point was trying to say that protocols should assume a variety of component failure modes.

I'd rather just have to trust the safety standards of the manufacturer of my car (and to a lesser extent the cars I might directly collide with), not the safety standards of every vehicle within transmit distance.

adamb | 7 years ago | on: Vehicle-to-Vehicle Communication Could Replace Traffic Lights, Shorten Commutes

I'm actually just asserting that systems advocating for use of V2V messages should account for misinformation.

That is, as a human I know that the drivers around me might have broken lights or misuse their signals.

I also assume that other drivers do not have my best interest at heart.

A reasonable V2V system should be expected to handle these scenarios just as easily as they handle wireless interference and packet loss.

adamb | 7 years ago | on: Vehicle-to-Vehicle Communication Could Replace Traffic Lights, Shorten Commutes

Nothing about my comment advocates for perfectionism.

My point is that V2V systems and research papers I've seen just don't make meaningful claims about safety. They instead make claims about convenience and efficiency, which are not substitutes for safety.

We know to test vehicles for crash safety before putting them on the road.

While I believe there are certainly ways to make use of V2V communication that increase overall safety, I haven't seen anything remotely resembling a crash test for V2V systems.

Creating a system that relies on the correct behavior of all components is not a recipe for safety or reliability. This is especially true as the number of components increases (this happens when a car exchanges messages with all the cars around it).

We need V2V systems that rely only on correct behavior of any component and allow for malicious behavior of some components.

The idea that a protocol version mismatch from some rolling deploy can cause injuries in cars made by other manufacturers is the sort of thing that I haven't seen a single person point out. How would something like this even be caught and debugged?

Advocating for an ecosystem where these things are likely but neither addressed nor considered is just plain irresponsible.

adamb | 7 years ago | on: Vehicle-to-Vehicle Communication Could Replace Traffic Lights, Shorten Commutes

Car manufacturers are extremely careful about the failure modes of components and sensors they put in their vehicles. Consider the design of the CAN bus, which has explicit support for failed and misbehaving peers.

I have yet to see these sorts of considerations in any V2V communication system. The idea that my car might act on (or propagate) incorrect/sabotaged information that it receives from a "peer" is a terrifying form of fragility.

This sort of failure mode needs to be studied and addressed directly before any of these systems are deployed.

It should also be assumed that these new patterns of information propagation will create a huge financial incentive for people to sell after market modifications that exploit the trust models of these protocols.

adamb | 7 years ago | on: Wanted: ‘Lost Einsteins.’ Please Apply

Hi @danicgross, glad to see you're iterating in this space! :)

It feels like your comment about the network and community being the most valuable part of program is probably right. Given that, is it possible to be a Pioneer, but not accept the money?

adamb | 8 years ago | on: Status update from the Reproducible Builds project

Beyond providing security, reproducible builds also provide an important ingredient for caching build artifacts (and thus accelerating build times) across CI and developer machines. They also can form the basis of a much simpler deploy and update pipeline, where the version of source code deployed is no longer as important. Instead a simple (recursive) binary diff can identify which components of a system must be updated, and which have not changed since the last deploy. This means a simpler state machine with fewer edge cases that works more quickly and reliably than the alternative.

I'm very grateful for the work that this project has done and continues to do. Thank you!

adamb | 8 years ago | on: Show HN: Get Paid to Build Your Next Side Project

Cool seed for a community! Seems to lack a place for discussion about submitted ideas, so I'll follow others' lead and discuss here.

The "industry specific deep learning" project is similar to something I'm working on right now. Though I'm not planning to charge for it.

To any folks here interested in this: Are you looking for a tool to get started in ML or for a resource to apply existing ML knowledge to a specific (possibly new) domain?

adamb | 9 years ago | on: Decommissioning Otto

Kudos to HashiCorp for realizing that the complexity of the project was getting away from them and for having the guts to pull the plug in such a public way.

adamb | 9 years ago | on: LambCI – A continuous integration system built on AWS Lambda

We suffered with a Jenkins-like solution for a long time before we decided enough was enough and we wanted to use an approach that didn't need as much soul-crushing, CI-specific effort.

If any of our experiences or insights can help others in their own environments, all the better!

adamb | 9 years ago | on: LambCI – A continuous integration system built on AWS Lambda

I forgot to mention that using the same build tool for both CI and developers has a number of other advantages, including artifact caching. When a developer downloads a change, their build will pull artifacts from the caches that build slaves have populated. So in many cases new changes (once deployed) are only built once, across the whole company by the build slave that kicks off the deploy. Everyone just reuses that cached output.

It was this sharing of artifacts that provided some of the impetus to use a sandbox, since a polluted output could poison the cache in hard to detect ways.

page 1