Information Systems Architect should write solid code, Part 2, how?

Code is the most efficient way to convey ideas to programmers. Architecture code should be embryonic, a starting point for a solution. Architecture principles help when they define some ground rules for composing solutions out of components, the boundaries and contexts of these compositions and the special conditions that should be guarded. All this can be expressed much faster in wireframe code, than in complicated text. If you can express architecture principles in diagrams and text, be sure to also express them in solid code. I call this say it in code.

In an earlier post, I stated that an Architect should write solid code. I’ve also tweeted about this, pointing out that an IT architect who doesn’t code cannot be trusted. To illustrate my point, I chose a fictitious test case organisation, Trotters, where Kermit works as an Architect. I described some of the consequences when someone like Kermit is far removed from code. If you are interested in this discussion but haven’t read my earlier post, I invite you to do so here.

In this post I will discuss some of the ways that Kermit could write code to make the architecture practice more effective in his organisation.

To begin with, here are some more (bold) statements – I could always prove the points at a later stage.

  • If you can draw diagrams, you could also provide framing code base that realise the diagram
  • If you cannot create code representative of the diagrams that you are drawing, then either:
    • you are unsure how to do this (no further comment on that), or
    • your ideas are not fully fleshed out yet (so you are not done), or
    • you are reluctant to get into the details, for some reason
    • In any case, this is a problem.

Writing wireframe code goes much faster than writing long documents, and it harmonises interpretation much more efficiently. Text and diagrams leave a lot of room for interpretation, every programmer is likely to interpret things his/her own singular way.

Code is the most efficient way to convey ideas to programmers. Architecture code should be embryonic, a starting point for a solution. Architecture principles help when they define some ground rules for composing solutions out of components, the boundaries and contexts of these compositions and the special conditions that should be guarded. All this can be expressed much faster in wireframe code, than in complicated text.

How should Kermit go about this?

To simplify I will focus on web application as the solution domain for Trotters. Kermit and the team are working with object-oriented (OO) design and development concepts, they standardise on UML notation. They could be programming on Java or .NET platform, that doesn’t matter here.

Here is a simple guidance for Kermit:

  • UML supports drawing component diagrams. Kermit has probably designed a layered architecture diagram. Therefore, Kermit can create equivalent UML artefacts in code by designing classes for relevant components using the most appropriate realisation path.
  • Components in a layered architecture will typically interact via interfaces. Kermit can create (mostly one-to-one) interface classes between each connection points in the architecture. OO design skills are necessary to do this right. This is a first test of Kermit’s ability to say it in code. This can be further refined when for example using a concept such as design by contract, of which ample free resources can be found on the Internet
  • Boundary conditions and constraints: modern programming languages like C# or Java offer an array of features to deal with constraints. (similar concepts are fairly trivial in other languages) In addition, boundary conditions and constraints can be expressed in test harnesses, in turn included in the codebase as part of a continuous integration setup. Such productivity measures are invaluable and they often teach teams aspects of the solutions that might not be trivial, let alone described in text documents.
  • Enforcing architecture: this aspect is made trivial when considering the development environments (IDEs) available to programmers these days. These IDEs ship (or can be complemented) with code analysis and validation tools. The most recurrent patterns and errors can typically be verified with little or no effort, thanks to the richness of the validation rules that are available for these tools. An organisation like Trotters, as discussed in part 1, is typically weak in this area, and that is a shame.

Once expressed in code, the following become enabled:

  • Efficient two-way feedback on the architecture: programmers can immediately spot any issues and raise them for resolution with the architect. Kermit’s ability to communicate in code will come in handy, as he is able to grasp implementation issues quickly and can improve his design in the process
  • Enforcing architecture principles: nearly every modern programming environment offer tools for validating code and testing boundary conditions.
  • Platform constraints are immediately brought to light: with architecture code, architects and programmers are immediately confronted with infrastructure constraints. This makes it possible to analyse the situation, convey feedback quickly to all stakeholders before anything is really built. This helps in reducing gaps in expectations across all groups.
  • Guarding against basic mistakes or repetitions: there are elements that are typically common to every solution in a given business domain. There is no value in painstakingly recreating such elements, as that would only expose Trotters to unnecessary mistakes (typing, errors or omissions due to human factors, learning the hard way).

An easy argument could be to claim that Kermit, as an architect, does not have enough time to delve into programming details. Another easy argument would be to claim that should Kermit delve in code, he would be overstepping his role (micro-managing?), or that the programmers’ creativity could be hampered. Such arguments are easy to bring up, it is very tempting when one knows nothing better to do. But such approach is misguided, in most cases. First of all, I argue that an architect can write solid code much faster than he/she can create useful documents with legible text and diagrams. A second reason why I think this is misguided, is that the tools and techniques available today are amazingly powerful and simple to use, not using them to their true potential equates to disservice to the organisation. As a way to illustrate this last point, I’ll take a totally unrelated example: as I look over the shoulder of some people using Microsoft Word, I’m often amazed to see how they go through the pain of manually formatting documents by inserting spaces and page breaks. These people don’t seem to be aware of features that were in the product since version 2.0! (the first I’ve used). That is the point. And this example is actually nothing compared to the inefficiencies caused by programmers and architects enforcing early 90’s work processes with 2010 generation of tools and techniques.

To summarise, if you can express architecture principles in diagrams and text, be sure to also express them in solid code to reduce communication gaps. I call this: say it in code. Users typically don’t relate to design documents (architecture is part of design in this context). Often case, design documents would pass approval gates without users noticing faults or shortcomings in them. And this should be expected, precisely because users hire architects for their solution skills and experience. Architects shouldn’t expect that users could validate solution architecture, which is what normally happens a lot – again I’m using the term users in a broad context here. On the other hand, once solutions are shown to users, they can more effectively provide meaningful feedback. So, Architects should express technical solutions in solution, and code is the closest form that can be less immune to human interpretation induce deviations.

So far, I’ve focused on the application architecture aspect of the Architect role, in my simplified view. Looking at the broader architect role, including business analysis and infrastructure design, saying it in code will involve other aspects that I won’t analyse too much here for a number of reasons. This posting is already quite long, I’m still trying to keep it short enough yet complete in its illustration. In future postings, I will tackle these other architect roles.

In the next instalment I will explore the limits of this concept. If I get more time, I will develop the concept further for Trotters by showing specific examples of the type of code that Kermit would be writing to help his organisation (eat your own dog food idea).

Nomad Computing as ultimate scalability concept

Nomad people take their cattle around to better grazing areas year in year out. They are always locating the best resources and have no problems migrating constantly. I see the future of computing following a similar model, swarms of rudimentary computing units self-organise to deliver the best service at their point of consumption. Resources are truly allocated on-demand, scalability becomes transparent.

In my take of Nomad Computing, object-orientation reaches its pinnacle. The traditional separation of databases and applications and web tiers will become meaningless. Each of these concepts will become more ‘nomad’, programmers need not worry about them. Databases systems and application server systems would have to mutate into entities that can register themselves and join in on local computing resource hubs. Once they’re in they start to figure out the best ways to self-organise for maximum throughput, spontaneously creating clusters based on usage patterns. Writing applications for Nomad Computing would be easier: no more coding login windows or data access objects, such usage patterns will be retired. Instead developers would concentrate on object graphs they intend to create, this would encourage crafting and creativity. Operating System platforms will also become largely irrelevant and pushed further in the background, taking away arguably one of the biggest pains in custom application development: deployment concerns.

A possible drawback to Nomad Computing will be hunting and fixing bugs and removing malicious software. These problems would possibly become intractable. Governance and safety would also become horrendously complex since unforeseen outcomes would be commonplace. Perhaps all these disciplines would need to evolve in entirely new ways. Specialised software will need to emerge to provide answers and keep ahead.

I realise this is all far fetched but it just seems that we might not be too far from it.

In a way we’ve already started to see early implementations of Nomad Computing, Amazon’s S3 and Apache Hadoop are good examples in the right direction. Power grids in electricity distribution industry are perhaps the closest model I see as incarnating Nomad Computing. Once we’ve really figured out how to do Nomad Computing properly we would be in a position to leverage massively multi-core systems as they become available.

Modality obsolescence

Now that we have got all manners of multi-core and kernel mode programming I think modality should be on its way out. Few things are less irritating than an un-responsive computer, computers should always respond full stop. With GUI systems, modality is often the cause of computer freezes, regardless of the ‘root cause’ of the issue. It’s the lack of modality in Unix system command line interface that make them mostly manageable and more resilient.

In the early 90s Microsoft Windows programming involved creating well … ‘windows’ and ‘dialogs’ in C++. The same thing could be achieved with Visual Basic and various other development platforms. Dialogs could be modal or non modal. You relied on the underlying messaging system to orchestrate functionality between modules. The whole concept was fairly simple, the complexity really came from the high number of APIs and libraries to code against. With C++ the other half of complexity came from the challenge for programmers to write code that truly reflected what they really had in mind and what they really ought to know about the tools and platforms being used, a gigantic ‘expectation’ gap. Writing my first dialogs and seeing ‘hello word’ was an exciting moment. Microsoft Windows and most graphical user interface systems still build on the fundamental concepts of ‘modal’ and ‘non modal’ dialogs and windows.

Looking back I think modality’s raison d’etre was and still is to try and preserve the integrity of the data being manipulated. You wanted to be sure that the program’s context is in a predictable state before proceeding further. This is inherently a sequential concept that ought to be left behind soon. In a true parallel computing world I would expect hardware and software modules to be even more self-contained, able to ‘move on’ if some desired state was not reached. This should rid us of computers totally freezing under certain conditions. This might never happen with silicon chips based Moore Law abiding platforms. Perhaps nano technology would help if it departs completely from ‘old’ models. Off to learning a bit about nanotechnologies then. Who knows.

Often overlooked PC security challenges

A less talked about fact: PC security challenges often lie with a weaker link, the user indeed. Here are examples of why that might be:

  • [Microsoft Windows] update notifications are often ignored by users. Looking over people shoulder I’ve seen many simply click away without ever bothering to read call-to-action messages and never letting the software update install
  • Web browser security also rely on people reviewing SSL certificates prior to visiting a page, users routinely ignore such warnings and carry on anyway. In fact users would happily follow any URL they get, they rarely check what they’re clicking on
  • Lugging around “garbage” : anything that can be installed gets installed, often case never or rarely used afterwards. This is a waste of system resources, PCs become irremediably cluttered and potentially damaging software is kept around. Only a rebuild will remedy such situations.

Enterprise deployments often remedy these risks by locking down PCs and forcing users through ever lasting roaming profile upload/downloads. Let’s get heavy handed and deprive people of their “liberty”. I’ve seen login and logout processes taking up to 10 minutes to complete, that’s insane! It gets even worse when using systems management software that jump in willy-nilly and start downloading huge software upgrades while you’re trying to get on with your work. Clearly you are working for your PC, not the other way around. If managers would calculate the productivity loss due to such soviet-style systems they’d have a fit. The next frontier in enterprise productivity battles is in fighting these clunky systems management software.

It seems as though people are pitching usability against security. Making users responsible for the security of their own PCs is probably as risky as leaving those systems wide open. This is not because people are dumb, it’s mainly because the whole notion of computer security and the tools of the trade are esoteric and pose totally unreasonable demands on users.

Good computer security starts with a good design, if it’s not build to be usable and secure it can never be properly usable and/or secure to use.

Parallel programming awaiting its John Von Neumann

Multi-core architecture is awaiting a decent computing model. This normally takes a generation to come to fruition. So far we’ve followed the John Von Neumann model, that’s how we naturally think: in blocks of sequences. This model suited us very well for a long time. No longer.

The big iron companies have been supplying multi-processor systems for ages. These systems were mainly designed for sharing expensive resources, time-sharing and similar models. We’re comfortable with that, design and develop programs to be run as smaller sequences of blocks, all well orchestrated and sharing memory units. The bulk of the algorithms available was conceived to exploit the Von Neumann model. Donald Knuth classics provided the bedrock for generations to program against.

Recently processing costs have come down significantly and chips manufacturers have been pushing out multi-core systems. We’re now stuttering to make a good use of these systems. What we’ve found so far is really a smaller and cheaper replica of the big irons’ model: it’s called virtualisation. Data Centre’s and CIO conferences buzz about this, it’s possibly over-hyped. It’s not surprising that we’re struggling to leverage multi-core architectures: the body of knowledge we live by is not really suitable for parallelism. Everything I’ve seen so far boil down (sooner or later) to arrays, loops and control blocks, some data access synchronisation, but that’s it. All of this is old news, Von Neumann-esque and is not truly parallelism.

Human beings provide a good approximation for illustrating true parallelism. When I take a step there’s a huge amount of activity (potentially a lot of coordination and collaboration) going on inside my body. I can’t begin to imagine how all that works. I’m sure some scientists could step forward and tell me a thing or two about it, but that’s going to be approximation too (current science as we know it). It’s tempting to think that a large amount of tiny steps are triggered and coordinated centrally within my brain. That’s the most logical explanation given what we think we know about human body and the role we think the brain plays in that. But it remains a guess, a very good one, the best we have. But who is to say that many parts of my body are not activated simultaneously? It it were we couldn’t possibly fathom the outcome, it’s therefore simpler to say that it doesn’t work that way. This is how I see the challenge with parallelism: an unknown unknown (not comfortable).

One reason parallelism eludes us is that we can’t figure out how to design and build systems unless we could predict the outcome and future behaviour of such systems. We might be reaching the limits of our current design ability. Where do you begin conceiving of systems that you cannot imagine how they will really operate? Is robotics a suitable field for this? Probably not quite, but certainly a good model to try and understand the challenges.

We can’t do parallelism very well because we’re missing Parallelism’s John Von Neumann, its Donald Knuth, its Grady Booch and so on and so forth. You can’t simply invent all of that so quickly, these great folks will slowly emerge, over time. Somebody will wake up one day and have an epiphany, many will flock in to hear more about the good news, marketers will learn new buzzwords, the whole cycle will repeat. That’s the way it is.

PS: I’m not a scientist, I conducted no surveys or lab analysis to come these conclusions. I’m just conjecturing, just some thoughts.

Google Mashup Editor illustrates what DSL initiatives ought to strive for

Google created a set of custom tag libraries and an editing tool, all this rely on back-end service through public API libraries. The construct is familiar enough, except that most organisations fall short of creating a solid toolkit correctly (the tag libraries and the development tooling part). Before I explain why I’m making a parallel with DSLs some preamble is in order.

DSL (Domain Specific Languages) is all the rage these days. The IT industry is addicted to acronyms, TLAs are the favourite (TLA = Three Letter Acronym). If you look beyond the hype, you’ll see that some of them do have value potential. One such TLA is DSL, companies on the fringe are focusing laser-sharp on it. The little i know about Intentional Software tells me that they’re very active practising DSL for their clients. It’s a smart move for any seriously ambitious company today, considering the amount of waste generated by IT. Just like people are worried about Global Warming, companies should be worried about Global IT Waste (GIW, my own TLA for today).

One of the rich tenets of DSL is that your teams would create a special-purpose language (a programming language a-la-IKEA if you like, but your very own) that befit the specific business you’re in, beautifully simple, flexible, functional, fit for purpose. Such language becomes the foundation for your programming toolset, hopefully shortening the turn-around time to produce valuable software while providing all the flexibility you’d ever want. Most of your focus would then be on building and maintaining glue-code, logic that binds your specific constructs with vendor solutions out there (vendor here include Free Open Source Software just the same). If you can imagine what I’m talking about here then this is the closest thing to software development nirvana.

Therein lies one of the problems, nirvana is not easy to reach therefore it’s just not trivial to execute DSL right. The clever move is to not so much see it as an end goal, but rather aiming for it as a continuous tuning exercise achieving greater levels of refinement steadily. Chances are that you and your organisation would reach your own nirvana and won’t even recognise it.

Microsoft has a lot of smart people that understood the potential for DSLs and they are cranking out tons of products and practices that help organisations with DSL. I am begining to see why they distanced themselves from the OMG’s UML evangelism, it’s a mind game that is best played with focus and purpose. It might not be very obvious to all today but I would bet that Microsoft’s efforts will ultimately achieve many of their goals [assuming they stay the course, not go revamping their frameworks and librairies every 15 months].

Really good DSL initiatives are set for the longer term, the value grows almost exponentially as the initiative matures. The reason is that it’s a tuning game, you don’t succeed in one step unless your business is simple or trivial. All of this is counter intuitive for many, most companies approach IT endeavours on a per project (hence per budget) basis and sometimes driven by middle management musical chairs. If you’re going to change executive direction/team every other year then there’s little point in adopting DSL. If you’re gullible product marketing would have you believe otherwise, fair enough, define “correctness”.

IT today has matured so much that really good programmers can achieve unheard of productivity. DSL works well for such people and the [preferably] stable organisations that employ them. If acronyms like BNF, LR, LALR sound strange to you then you’re probably not ready to seriously embrace DSL. Do yourself a favour, ignore DSL and your life would be easier. If you think that object modelling is enough to qualify what you’re doing as DSL then I’d say suit yourself.

One way to look at Google Mashup Editor is that they’ve created a kind of DSL so that folks could quickly create mashups with Google’s products and services. It’s likely to be addictive for those that try, it will make you think… So far I wasn’t buying too much into all the fuss suggesting that Google was challenging Microsoft. But Google has been steadily making many of the technical moves that makes me envy them. I have to give it to the analysts for their foresight, Google is on to something really big and they have the right approach to technology.

If you are one of those companies wondering why your IT teams struggle so much to deliver what you want, or that IT is “just not living up to your expectations”, I suggest you take a tour of Google Mashup Editor, find out about DSL as discussed in this context and tell me what you’ve learned. If you’ve read this and followed my advice I’d love to hear from you, right here. Go on then, I’m patient.

Plugin technology : a beauty that often turns nasty

Plugin technology is a universal driver for widespread adoption, dizzying success at times, even revolutions! But plugin technology is often the source of dreadful clumsiness and that utterly leads to the demise of the very thing it made popular in the first place. What goes around comes around. Sometimes.

I’m using the term plugin here loosely to refer to anything that can be called plugin, addon, extension, application or appliance. very ambitious, i admit. Something suports plugin if it’s possible to extend it beyond its original shape, content, purpose or use. If it is pluggable then someone will put it to some uses well beyond what the inventor or creator had originally anticipated. As i was analysing some recurring issues around me i came to this realisation. One day you go like “This is superbe! I can see lots of ways to make this better…” then some time later you find yourself thinking “Oh crap! I’m sick of this, i need something more basic, that just does such and such (instead of trying to be all things for all people)“.

Some examples of plugin enabled technology that have changed our world: human language, electricity, telephony, transportation (road, rail, air), television, cable television, Microsoft Windows Operating System, Eclipse IDE, Firefox web browser, i could carry on. When i talk about plugin for human language I’m thinking of dialects and other specialised uses of the language that deviate from its core elements. I must have confused my reader at this point, but think about it carefully. One can argue if spoken language can be called a technology. Obviously electricity and telephony continue to do wonders for us, but they have dark sides too.

A piece of technology becomes exciting if it has some form of plugin architecture. For better or for worse, everybody want a piece of the action, no one wants to be left behind. After all the innovators’ club is very exclusive with few members, the very large majority simply tag along and try to make the most of it, Rabelais’ sheep of panurge come to mind. But put these wonderful plugin technology in the hands of the masses, it won’t be long before the whole thing is totally messed up. If you take out the part that bolted-on plugins play in a technology failure, you will find that it is very significant.

Apple Computer must have realised this early, that could explain some of their policies. Recently for example when they announced the iPhone and said at the sime time that it won’t support third-party software. In this case any third-party software is a pluggable component for the iPhone, likely to break the stability and harmony of the device even at launch. No amount of testing would eliminate such risks, examples abound to support this hypothesis.

Some examples of plugin technology turned nasty: Microsoft Windows famous security vulnerabilities, pluggable softwares getting bloated to the point of irrelevance (Netscape web browser, Firefox more recently).

This topic is so rich that it could fill volumes after volumes. It’s really the story of how we learn (or don’t really learn from mistakes).

Look around you for examples of plugin technology turned nasty, you could be surprised.

Identity as a service

Identity as a service is an interesting concept that doesn’t seem to have catched on yet

Here is an idea (old news to me): Identity as a service

As i sat through a talk on identity-driven computing at Novell’s Brainshare 2004 in Barcelona, i really thought Novell was up to something big. A year on i haven’t heard much from Novell, now i see that folks at Losely Coupled have started talking about the concept of identity service. Indeed most serious computing solutions require some form of identity functionality. It should be natural that this is baked into the fabric of Internet-based solutions. Passport is Microsoft’s answer to this issue. The rest of the industry is making a lot of noise but no provider has come forward with this kind of service on the tap as yet.

Linus Torvalds Outburst Sparks Fierce Debate: Does Open Source Software Need Specs?

Specs are useful when written by authoritative people and with specific goals. You need specs to solve issues like interoperability, defining metrics so that all stakeholders are aligned on what was built (or what is going to be built, as is often the case).

When specs are aimed at some political agenda, then Linus’ point is relevant. I guess in this industry, specs have often being driven by political agendas rather than solving particular problems.

More on this topic: Linus Torvalds Outburst Sparks Fierce Debate: Does Open Source Software Need Specs?