Removing the Fight from Enterprise Integration

by Michael Figura

"Be not afraid of greatness. Some are born great, some achieve greatness, and others have greatness thrust upon them.” – William Shakespeare, Twelfth Night

Enterprise integration can be an absolute blood sport. No matter how much you prepare for the tides of war, only life and reality can ultimately serve as your teacher. Interfaces suddenly change without warning. Back-end systems go offline unexpectedly. EDI coordinators sip their coffee from mugs that say “Shit happens.” On good days only half your team is holding their heads, digging through spaghetti code written by a cavalcade of contract engineers who have long since left the project. On bad days half your team is dusting off their resumes and heading for greener pastures, because when production is down and business comes to a halt, it’s everyone’s fault across the board.

In the Navy they have a simple saying: if you see a fire, it’s your fire. That is to say, if you waste even a second looking around wondering who started the fire or how, the ship sinks and everyone dies. During my second week onboard the U.S.S Abraham Lincoln there was a fire in the aft galley. I’ve never seen sailors move so fast. They were running toward the fire, not away from it. In the face of great odds and danger, there’s simply no time for fear or chaos. Experienced individuals know that responding to these situations with the wrong attitude will generally end in much greater disaster and loss. We have to face these challenges head-on.

In sport fighting they have an acronym: CNC. Contact and control. In the ring contact is control. The closer you are to your opponent, the more contact you have, the greater the opportunity for controlling your opponent. Conversely, in enterprise integration we pride ourselves on loosely coupled interfaces that connect disparate systems, across organizational boundaries. Yet, we invite occasional information silos, just like we avoid intimacy between information trading partners. I’m here to tell you, this approach is part of the problem. We know confidently from our personal lives that the most successful relationships exist as a healthy balance of intimacy and distance. Keep your loved ones close, but not too close. There’s no reason why we can’t apply that same wisdom to our business lives, as well. Keep your information trading partners close, but not too close. Keep your external interfaces close, but not too close.

This is an easy enough concept to grasp, but what do these relationships look like in reality? The simple answer is… they’re based on trust. And as we all know, trust is earned and maintained. It’s not handed out lightly. Thus, a trustworthy relationship is earned through successful testing. And it’s maintained by trustworthy service level agreements. In other words, nobody should be coming to work with any surprises awaiting them. And when surprises do occur, an elegant approach to unexpected faults will always act as your last line of defense. Trust, as it turns out, is built and maintained by the responsibility of both parties to follow a few simple rules:

  1. Trust your testing
  2. Trust your service level agreements
  3. Trust your exception handling

Testing has become the forerunner of modern development approaches. Most development projects today implement test cases and the underlying test methods before any production code is ever written. This isn’t just true for Java, C# and other object-oriented projects. It’s true for enterprise integration, as well. But which test methodology should you adopt? One common mistake is to base your test methodology on the capabilities that come with a COTS test product, like Test Director or Postman, for example. Why is this a potential mishap? When this approach is taken the test methods often end up falling short of all real-world scenarios. In other words, you built a hammer, and you’re staring at a screw. Another common mistake is to choose a test methodology solely based on its industry popularity. Test driven development (or TDD) commonly uses the triple A approach of arrange, act and assert. Some systems are much more complicated, and this approach would require more individual test methods than any team member is willing to write. Sometimes behavior driven development (or BDD) is the only way forward. The folks over at Cucumber.io have written a great introduction article to help you understand the difference.

Trusting your service level agreement is an entirely different can of worms. After all, it’s an agreement. Both parties have supposedly come to an agreement, which means the SLA behaves as a form of contract. In the world of interfaces, it’s not absurd at all to say that the SLA is a contract about a contract. If the underlying interface changes in a way that is not defined by the governing SLA, both contracts have been violated, and trust is broken. So, how do we keep this trust from being tarnished? By managing expectations correctly. The SLA should clearly define how versioning and testing periods are implemented. When do new versions of an interface become exposed and usable? When do older versions of interfaces become deprecated and eventually taken offline? When and how does testing occur? What features are available during testing? And finally, how are exceptions handled?

In the world of enterprise transactions trusting your ability to handle failure is equally as important as trusting your ability to succeed. Remember, shit happens. And it happens all the time. The ability for a transaction to be durable depends on an agreement between both parties that a retry will occur, and it will occur a predefined number of times before the exception is escalated to another level or state for further investigation and handling. Too often do companies approach exception handling on their own terms, and from the point of view of black-box architecture. This is a mistake, as it leads to surprises. Thorough testing should lead to better exception handling. After all, if you don’t know what kind of exceptions you’re expecting, how the heck do you handle them? Don’t treat your consumable service interfaces and information trading partners as opponents. Treat them as friends. At OAGi we have a favorite new saying: Keep your opponents close. Keep your friends closer. Good luck, and we’ll see you all ringside.