[brianwillis]

This might not be the hardest bug I've tackled, but it certainly took the longest to solve.

It was the early days of SOAP, and I had been assigned the task of integrating my employer's software with a third party's, so that the applications could share data. This third party org was a wealthy, powerful mega-corporation; and my employer was, well, not. The third party produced a spec for the interface, expected us to follow it, and offered no help from there.

I built a solution. It worked on my machine. Solved the problem. All was right in the world.

I moved it to the test environment. It worked again. Demoed it for one of our customers, and everyone was pleased.

Deployed it to our first beta tester. One lonely employee working accounts receivable, tucked away in the corner of our customer's office.

It crashed.

I checked everything. I mean everything. There are still particulars of that little Windows 2000 workstation that I can describe vividly. Which programs were installed, which patches were installed, how Windows had been configured, how the firewall worked, I even got permission to install a packet analyzer. My employer only had a handful of customers, and the beta test machine was near our offices, so I was over there personally a lot over the following weeks.

We brought in the customer's network support people. They found nothing. They could see the packets leaving, and an error coming back, but couldn't offer more than that.

We brought in the best networking engineer in my company. He was stumped.

What really shook my confidence was knowing that competitors mine had gotten this interface working. This wasn't some half baked project that I could blame on someone else. Others had succeeded where I'd failed.

I practically had to walk across broken glass to get on the phone with the third party's development team, but with enough pestering I pulled it off.

The phone call involved me sitting at the beta test workstation and firing off a request so that they could view it hitting their servers live. The developer who I spoke with immediately spotted the problem.

You see, when you send a SOAP request, you send the date and time that you're making the request along with it. The clocks on the client and the server were too far out of sync, my requests appeared to be coming from the future, and so the server disregarded them with a blunt error. Interestingly, the workstation clocks at my company's office weren't too far out of sync, which is why it worked in one place and not another.

Stuff I learnt:

1. Third party interfaces require a point of contact at both organisations who can talk with one another. This is non-negotiable.

2. If you send an error message that reads "Error", you're a bad developer and should return your computer science degree to your university and demand a refund.

3. No matter how well written the spec is, something always gets left out.

4. Persistence maters more than anything.