The 3 Most Powerful Architecture Decisions (10 min podcast)

[I am experimenting with what I call an 'enhanced' Podcast. The audio is accompanied by visuals and the complete transcript (below). So the reader can listen, watch, or read, as convenient.]

Today I am going to talk to you about the three most powerful architecture decisions. It will make you an expert in finding and dealing with the most critical solution problems first and well.

A little backstory: I transitioned into architecture in 2001. My first substantial project was a dealer incentive management system for a large company. There was a lot to work out, and it was daunting. Where would I begin? How would I know if I was making the right decisions? I had the formal architecture training, but you know what, unless one gets into the deep end, it's all theory. I completed it somehow.

But it was only after doing four or five architecture projects, that I started working things out more easily. I began to see the patterns in the problems and solutions. And I discovered efficient ways of getting good outcomes.

Today, I will share with you the technique I refined that will demystify a critical aspect of architecture for you - how we can find and expertly deal with architectural problems. It can seem mysterious to outsiders. But it is not an art, and it does not have to be a struggle. Let's see how we can make life easier for ourselves and deliver fine outcomes for our customers.

Preparation

Now, if you look at a typical IT landscape of systems and how they work with each other, the complexity can be enormous. Imagine scores of systems and hundreds of interfaces. If something new or a significant change comes along affecting a large area, how does one start dealing with the large number of possible architecture decisions required? How do we even know if we have asked and answered the most important questions?

It can be a jungle, and we need to slash a path through it. We will start with these three tools.

• First, a clear destination: which is to say, clear functional and non-functional (or quality) requirements.
• Second, a birds-eye view of the land: meaning the details of the existing architecture in the form of lists of assets, their functions, integrations and diagrams.
• Third, a machete: that is a clear and sharp architectural thinking technique.

[This includes keeping Design Decisions (DDs) and Technology Decisions (TDs) separate from Architecture Decisions. How they differ is beyond the scope of this Podcast. Please message me personally if you want to know.]

Now that we are well equipped, we can clear a path through the undergrowth in three stages, in the form of three types of Architecture Decisions (aka ADs). Why three and not five or ten? Well, this is from my experience, from getting through such jungles a few hundred times till now. I have learnt that these are the most critical problems, and if we deal with them first, we make headway fast and get control over the other Architecture, Design and Technology Decisions. Trust me and try it. It works for me, and it will work for you.

The 3 Fundamental Architecture Decisions

1. Function Placement ADs
2. Data Source ADs
3. Integration Method ADs

Let's have an in-depth look at each. Now keep this in mind, we will be applying the core architectural motivation –

Achieve the maximum functionality and quality at the minimum long term cost. It is the 'Science of the Optimum'.

The underlying patterns that help to get this balance are Cohesion, Loose Coupling and Reuse. They drive several services, data types and interfaces to the same component and separate others. [You can look up these three essential concepts in books or on the internet; or drop me a line and I'll explain them.]

Problem 1: Function Placement (FP) ADs

They address the central problem - where should the functions be executed? To work this out, take each Functional Requirement and see if there is an existing system, component or sub-component (these formal architectural constructs are the subject of a separate article), or a combination, that provides the underlying technical function. If not, see if there is one that you can extend for it. No joy? Then you need a new architectural component for it. Reusing systems is the least expensive overall. Creating a new one is the most costly and to be avoided if at all possible. And for a new one, buying may be cheaper today than building. Repeat this AD for every new or modified business requirement.

Here is a mantra that you can apply while attacking Function Placement ADs: "Reuse before modify, modify before buy, buy before build".

Problem 2: Data Source (DS) ADs

The second problem that we deal with is - from where will the function get the external data it needs?

Information Technology is in essence "data in - data out". And the overall enterprise cost of IT is pegged to the amount of data it handles and stores. Storage costs per Petabyte are falling, but the need for the enterprise to reduce costs will always outpace everything. So, we architects have to take the utmost care to optimise the flows and storage of data.

There are two main types of data stores - those for Systems of Engagement (SoE) and those that are Systems of Record (SoR). An SoE does not need to store data for very long, whereas an SoR does, for regulatory and analytical needs. Typically, SoR data stores can be ten to a hundred times larger than SoE data stores.

Now, for an application that needs to carry out a function that needs data that it does not itself generate or 'own', the architectural problem is whether it should (a) take it from the external owner, let's call it the 'master' or (b) some other system that already has a copy of it, let's call it the 'record' or © make a 'local' copy for itself.

There has to be an excellent reason for making a local copy of some other system's core data because every copy involves computing and temporary storage costs at the source, network costs to transport it, and computing and storage costs in the destination system. A few of the right reasons are - low latency requirements, information enrichment that is owned by the consuming application, and a retention requirement that is longer than provided by the master. Take these factors into account while making this AD.

Here is the mantra that you can apply while attacking Data Source ADs: "Master before Record before Local".

[I have a litmus test that quickly and objectively indicates whether we should use the external master/record data or a local copy. I would like you to focus on the steps in this Podcast, so I will leave it out and publish it in a dedicated future article. You can drop me a line if you want it right away.]

Problem 3: Integration Method (IM) ADs

Having got so far with the Function and Data Source ADs, the edge of the jungle starts coming into sight, and Integration architecture decisions will get us through. They are relatively straightforward, but not any less vital. It addresses the problem - how should a system talk to others to get or give data and services?

The focus is to ensure simplicity in communication between the many components in the overall architecture as the cost here comes from maintaining the resulting mesh of connections. Direct connections between systems are the simplest, and it is possible in a mature Service Oriented Architecture. Here, well-designed clients and serving applications can take care of integration needs such as standard protocols, publish-subscribe patterns, peak load smoothening, data aggregation, etc. Only where this maturity is not available should we use Enterprise Application Integration (or EAI) platforms to provide the necessary facilities. Finally, an old fallback is to exchange the required information between applications using files. With file exchanges, there can be data reconciliation concerns and additional processing, network and storage costs. But for certain problems, it is a natural choice. It is suited for larger data sizes, higher latency toleration and batch operations and used more often even now than one may think.

Here is the mantra that you can apply while attacking Integration ADs: "Direct before EAI, EAI before files".

[For this too, I have a litmus test for deciding between direct and EAI mediated integration, but like the one for data copying, I will leave it for a dedicated future article. Please contact me if you want it immediately.]

Record the ADs

As you progress through the AD jungle, you need to put down a map of your way through. So, record the above ADs in a template. It should have the AD ID and Type, a concise problem statement, the solution options considered, the decision, the reasoning and any implications. Mention the related ADs, TDs and DDs as they become available, for cross-reference. Filling this template will help show others why you took a particular course, and they will be able to follow you easily.

AD Record Template

Endnote

These three ADs of Function Placement, Data Source and Integration Method apply to the problems of all the architecture disciplines, but especially to business, enterprise and application architectures.

Of course, there are several more domains - information, integration, infrastructure and security. To cover all the seven fields and their layers for the many IT systems could mean scores of problem statements and decisions.

But with the above approach, you will break the back of the work and deliver high value fast for your customers.

Enjoy, my friends.

---

Copyright Shashidhar Sastry, 2021. All rights reserved.

Join my email list - it's easy to unsubscribe if you change your mind.

quality-thinking.com