What flavour of compromise?

So, dear Bob, I think the answer is fairly simple. If you can’t build it all and you can’t buy it all, you simply have to find a compromise. The only question is what flavour that compromise should be.

The problem with many compromises is that they occur in the wrong way – as the accidental outcome of a failed effort to either buy all or build all.

To make choices which are not accidental requires that we open our eyes and accept the reality of the need to compromise, and on this point both developers and architects alike struggle to get real. Without the ability to put our minds in neutral and to analyse the situation from a point of rest, there is a huge chance that we will assume a silver-bullet approach to technology can work. Technologists get excited about technology because it’s powerful. We understand the secret potential inside these machines and we yearn to see that potential realised in the lives of the people we work with. Any whispered message offering the hope of quick realisation (rather than the more usual drawn-out frustration) is instantly compelling.

So perhaps the better path is to see that for any given problem, the likely solution is a combination of buy and build. Not the horrific buy-then-customise option which is so often the way things work, but instead, analyse the problem in terms of complexity, buy the complexity, and build the rest of it.

If you’re in the business of producing cars, buy the engine and the chassis, and possibly the fiddly electronic bits, then make the rest of it.

There’s a reason behind this apparently arbitrary designation. It comes down to knowing what our core values are as producers. If I am a car producer, the equipment involved in the manufacturing of an engine and/or chassis may be quite distinct from that of manufacturing the seats and body. In order to concentrate excellence, focus on doing one or two things really well, and outsource the rest. That way you can continue to add true value without diluting the excellence of the other components.

This can be applied using a workflow as follows:

1. For any given build, when you break the build down into its component parts, break them down into components which are of roughly equal impact to the end-user. Or, to state it another way, each component you identify should be of roughly equal business-value.
2. Decide on an aspect of the project which usually is the most painful (for most people this will be either "effort” or “complexity”, but it may also be an aspect which is simply outside of your core expertise (e.g. “user interface”))
3. For each component, assign a pain rating (3 – high, 2 – medium, 1 – low, 0 – none)
4. Try to buy components for anything 3 or 2, try to build anything 1 or 0

Bob's dilemma

I know what Bob means. BizTalk is, no doubt, a huge powerful beast which I know can be a really useful tool for traditional EAI. I suppose where it falls down is as a backbone for your entire service architecture. I don't think it was ever architected for that purpose. I think what's happening is that Bob is starting to think of things from a business-as-usual perspective. It's hard for me as a developer to think that way, but I can understand where he's coming from.

The dream of business technology architects is of this world of smooth-running, purchased off-the-shelf systems which they know will work (or be fixed) because the internals of those systems is Someone Else's Problem. As the teacher once said:

An SEP is something we can't see, or don't see, or our brain doesn't let us see, because we think that it's somebody else's problem.... The brain just edits it out, it's like a blind spot. If you look at it directly you won't see it unless you know precisely what it is. Your only hope is to catch it by surprise out of the corner of your eye.

The technology involved in making something properly invisible is so mind-bogglingly complex that 999,999,999 times out of a billion it's simpler just to take the thing away and do without it....... The "Somebody Else's Problem field" is much simpler, more effective, and "can be run for over a hundred years on a single torch battery."

This is because it relies on people's natural predisposition not to see anything they don't want to, weren't expecting, or can't explain.

I don't think this is all about wanting to avoid responsibility either. And it's not really about control. Yes, it would make things simpler to be able to control everything, but it's more than that... There's actually a sense of embarrassment or disbelief about how hard it is to get past the basics. I think these architects are really longing to do something wonderful with their technology, but they just keep getting hung up on all the detailed problems that come their way. Not least among these is the politics of dealing with all the people involved. It requires a certain amount of trusting the unreliable developers to create a custom built system and that just seems like a poor way to powerfully execute on the task at hand – it’s got way too many points of failure. Technology should be simple, sleek, solid – not flimsy and byzantine and likely to blow up if you look at it the wrong way. Technology should be built by Buddhists…

The other thing is – and an architect knows this better than most – often it’s not what inside that counts. Good solutions exist which have an amazingly sophisticated and beautiful interface, with some one inside peddling a bicycle to make it all happen. I think architects instinctively feel that in order to get a really satisfying solution, a sacrifice is required, which is to live in the knowledge that inside, everything is held together with tape….

So I understand – I really do get what he’s saying. But I suspect that he’s still way off base here. There is no way I can look at BizTalk and say, “yep – that’s the ticket to a successful enterprise. Just run everything through that box and everything will be ok.”

So what advice to give? I suspect another post is in order. The problem is pretty well described, but I still don’t see the answer. I understand why “just build it all yourself” isn’t a viable option, and I can see why “just buy it all” also doesn’t work.

And it gets worse!

Because I didn't blog on Tuesday, December 13, 2011

So I just got off the phone with Bob! He's worried that his soul is destined for damnation because he found himself thinking maybe Biztalk would be a good idea.

I paraphrase:

"I remember thinking when I first worked with Biztalk that it was the biggest pile of s**t since I tried to implement .NET SSO using Tiovli. And now, I find myself thinking.. hmm, yeah, we could use Biztalk here - avoid having to do some custom development. What the hell is going on? I know Biztalk is rubbish for anything nuanced, and yet, in this new position, it suddenly makes sense. But... I'm sure I'm missing something. It's like my soul has been corrupted."

Mwhahahaaahahaaa....

But seriously that's not good. I need to have a hard think about this. How to save Bob's soul?

What happens when a developer becomes a manager?

Because I didn’t blog on Monday, December 12, 2011

A friend of mine has recently moved into a new position at work (insurance) where there is little day-to-day development, and more focus on communication with the business about strategy and e.g. PoC work for frameworks and vendor platforms.

He says that he can already feel an internal change happening. Things that he observed with horror in his immediate manager he now finds himself doing with increasing frequency.

So I asked him a bit more about his immediate manager

He said that his manager was recently made the director for all of IT for all of Europe in their section of the company. It’s not a huge company, but it’s still quite a big role, and this manager is moving directly out of a architect role (having shortly before moved out of a senior development role). He says that he used to see his manager analysing things with a “wrong-headed” approach. “He just doesn’t seem to engage with the idea of a technology on a level which makes sense. When he looks at an idea, he’s thinking about it in the most horribly over-simplistic terms. He’ll make judgments about things which are based on completely flawed understandings of the basic concept.”

To which I replied, “But, isn’t that just a difference in perspective?”

“Yes, that’s what I’m saying – I am starting to see that attitude in myself now and it's due to the perspective. I can tell that I’m thinking about components in larger blocks, and that things like efficiency and uniformity is becoming more important to me than the technical detail because those things seem like a waste of time if I can just buy off the shelf. But I remember that this was a big problem when I saw how my manager responded to ideas in the past.

If I remember correctly, I perceived my manager making decisions that would produce what I thought of as very sub-standard outcomes (in terms of client experience and technical soundness). I was being asked to be a developer when, in reality, there was no real inclination on the part of the manager to develop anything. For him, development was the most painful, risky option, and now that I am moving into that kind of role, I understand exactly how he felt. But it still bothers me that I am betraying the reaction I had.

Perhaps the problem is that we just don’t make a clear enough distinction between an off-the-shelf product and the decision to develop a custom solution (or the boundary between both). It makes me feel even more strongly about the dangers of the “middle road” which is to buy an off-the-shelf product and then customise it to the business’ needs.”

My next question: “So, can ever see yourself justifying the development of custom solutions again?”

“To be honest, even if an off-the-shelf component is less than perfect, that now seems preferable to me than custom development. I think that there’s a role for development in companies, but it should be considered very, very carefully. I think that there truly are gaps in the market, where you still have to build it yourself. The main problem is that I think we under estimate the cost of developing custom solutions to the level of maintainability and operationalization required for it to be the right choice. I believe that you should develop components which are less than perfect only when you can prove to yourself that there is time and capacity to do an really, really solid job of the development effort.”

MEP diagrams in SOA

Because I didn’t blog on Friday, December 9, 2011

So, after searching the Thomas Erl cathedral (www.soapatterns.org) I can’t really find anything focused on the message exchange pattern. Of course, for SOAP, the MEP is all tied up with the WSDL (“Port type”, isn’t it… it’s been a while…). Basically you get a one way, duplex or reqest-response pattern. But, for the most part the diagrams seem to focus on things like:

Service composition

Service composition and internal structure

API (even when the pattern is about messaging)

 

So I gave up and headed over for MSFT to see if there was anything used to document the WCF world. But not really any joy there either. There are a few bits and pieces but no standard approach to capturing the messages:

Explaining a duplex pattern

Thinking about the service internals again

I suppose, after all, it’s “Service” Oriented Architecture – not Message oriented. People who are doing it seem to just stick to the Sequence diagram.

So much for that.

Digestion underway

Ok, so here’s the diagram again:

In the first column we can see the controller swim-lane. The Total Request message object drawn in this swimlane is constructed by the Controller.

Arrow (1) shows that the Total Request is passed to the Invoice object (which has a black dot for receipt and/or dispatch of a single message object)

Arrow (2) shows passing the Total Request to one or more Line Items

Arrow (3) shows passing the Total Request to the Tax Calculator

This captures the message flow, but it doesn’t show the details of the message. Nor does it show how each object interacts with the message.

Is that important?

Yes – the message is essentially its content. The semantics are embodied not only in the name type of the message, but also in the content of the message.

I’m calling this a message content diagram and the idea is that it supplements the message flow diagram (above).

(1) On the first line we can see that the Controller sets the Amount to 0 before passing the message to Invoice

(2) When the Invoice passes the message to the line item(s), the Amount gets updated by adding the line item amount

(3) When the Invoice passes the message to the tax calculator, the Amount gets updated by adding the tax total which is a function of the original Amount

I think this works quite well. I’m still not showing the public method names, but there’s a suggestion of the business logic within Tax Calculator: calculating the tax total is a function of the amount in the message

Now that I’m messing around with this, it occurs to me that the SOA guys already have the idea of MEP (Message Exchange Patterns). I ought to check-out what Mr. Erl’s been up to with his diagramming stuff. Tomorrow.

Message sequence diagram

So after yesterday’s browse around the various UML diagram flavours i could find, I keep coming back to the sequence diagram as the one which is most accommodating to the tracking of messages through a system.

In fact the only problem with it is the imbalance between the messages (treated as method calls) and the actors (the “real” objects).

Starting with a bog standards sequence diagram, here’s a sample to work with:

We have a controller, which issues a command to calculate the total due to an Invoice object. The invoice object queries each of its line items and builds a sub-total before calling the tax calculator and calculating the total to return.

So, I’ve thought up something I’m calling a message sequence diagram as a starting point. Here’s what I’ve got so far:

 

As you can see it’s quite compact. Part of this is a result of the fact that I’m passing a message object around “Total Request” and the method names (and return from calls) are not shown. The message is referenced by each actor upon receipt of the message.

Message delivery is shown by the arrows and the sequence of deliveries can be determined from the numbering where necessary.

Now I shall digest this for a little while.

Diagramming message flow

So today I decided to look to see whether I can find ways of designing systems from a message-oriented point of view.

First strike against the googlebase was the UML2 Communication Diagram (Scott Ambler’s site – agilemodeling.com)

So let me take a look at this for a moment. How many ways do I hate this diagram?

Well the good part is that you can see a sequence of communication. But that’s where it stops. It’s very granular (see the label about “series of getter invocations”). The communication is about methods called – not about messages exchanged. But hang on – I’m forgetting my OO roots – methods are messages – or the poor remnant of what messages were originally intended to do.

So in a sense, the messages are shown there, but only in miniscule detail. This diagram shows meta-flow, which you definitely need for message-oriented programming, but it’s still not message-oriented because there isn’t enough detail in about the messages. There isn’t much detail in the object boxes either, but the focus is definitely on the objects.

Next up then, (as recommended by Steve), the humble Sequence diagram (of which I am a big fan)

Ok, so here we have a nice clean diagram showing the flow of messages between instances of objects. And I very much like this – it puts a lot of focus on the messages themselves. As much as 75% of the diagram space is message flow. Of course, the main highlighted units are still the object instances themselves, and their lifelines. But it’s closer…

And I suppose the problem is that much of what I am seeing is about flow and sequence, not about the messages themselves. This is because the messages are seen as “belonging” to the objects (i.e. the “methods” belong to the class of an object). In reality, Dr. Kay’s take on messages is much more independent from the “receiving” objects themselves. To capture his principles, we would have to have a diagram which treats the “receiving” objects as equivalent to the “message” objects.

That gives me an idea! But enough for today. Till tomorrow. Perhaps Lee will have blogged by then.

Betrayal

So it's been a week since I started blogging and bloody lee hasn’t blogged at all. So while I was getting insecure about my technical underpinnings, he was calmly blowing smoke up my ass about his. Well I’m not going to let that grind me down.

The “OO is a hoax” thing

I tend to cringe when I hear this particular debate, but I grin and bear it because it often ends up touching on a subject which I am much more passionate about: messaging.

Alan Kay’s stuff on messaging (see here on the brilliant c2.com) highlights the following top-two principles of OO:

1. Everything is an object
2. Objects communicate by sending and receiving messages (in terms of objects).

-------

Now, thinking about the garden variety C# (or even javascript) code I have to work with on a daily basis, the general idea is:

where the big circle bits are the “objects” which pass data (“A”, “B” and “C”) along the arrows. This diagram is quite UMLish in its conception, and it highlights the “objects” as the important bit.

If we think of circles “3” and “4” as being instances of the same class, then we could also draw a class-focused diagram like:

--------

But based on Dr. Kay’s principles above, perhaps we should think more like:

Here we see the messages “A”, “B” and “C” front and centre (and I think I mean message instances, not message classes), along with their usage pattern between instances of objects in the domain.

Hmm. This might be clearer with better diagrams. I’ll work on it.

Well here goes

I had a stab at updating the qunit code. I started with the following test:

  1: module("normal tests")
  2: 
  3: test("that equality works as normal", function () {
  4:     var a = "astring";
  5:     var b = "astring";
  6:     var c = "notastring";
  7: 
  8:     equal(a, b, "strings are equal");
  9:     notEqual(a, c, "non equal strings are not equal");
 10: })
 11: 
 12: test("that ok works as normal", function () {
 13:     var isIt = true;
 14:     ok(isIt, "yes it is!");
 15: });
 16: 
 17: test("that pending is implemented correctly", function () {
 18:     pending("Something I have yet to complete");
 19: });

Which produced the following output:

And after some work, I achieved (the orange looks yellow on this blog theme, but oh well):

But! I had do change quite a lot of code and CSS in order to achieve this. I’m not sure I like the resulting code. I may try to do this some other way…

Anyhow, the updated qunit js and css files I ended up with are here:

• https://raw.github.com/goofballLogic/qunit-edit/master/qunit.edit.css
  
• https://raw.github.com/goofballLogic/qunit-edit/master/qunit.edit.js