Technical Debt, Target States and Local Maxima

There’s usually a tendency to select least-cost solutions incrementally, which typically means tweaking legacy estate rather than doing anything more transformative. While this does minimize delivery risk within individual projects, it can shift risk onto the institution’s future and allows it to cumulate until the “burning platform” moment requiring drastic, large-scale measures with no guarantee of success.

There needs to be a better way to weigh the pros and cons of solution options, balancing the short-term and the long term, the tactical and the strategic.

Technical Debt (TD) is a good framework to help determine relative merits of different solution options, if it is defined as the spend not accruing to the strategic target state of the assets being changed.

I like a geometric explanation:

C is the Current State

T is the Target State – the true, uncompromising target one should aim for long-term – 3-5 year horizon 

r is the funding available, circumscribing potential solution space – anything implementable has to lie within the circle of radius r around the current state 

Solutions 1 & 2 have costs r₁ and r₂ and while 2 is cheaper (r₂ < r₁), they have radically different Tech Debt:

s₁ is the strategically-aligned portion of 1, and its Tech Debt TD₁ = (r₁ – s₁) - regrettable spend that does not accrue to the strategic target. 

It’s clear that s₂ (being the projection of r₂ onto the line C-T) is actually negative, so solution 2 is a net detractor from T

Minimizing Tech Debt simply means maximizing alignment of next state solution with long-term target state.

The reason why target state needs to be uncompromising is so that it can serve as a North Star to judge alignment against, or an etalon to weigh options by. Even if it’s not achievable for some years, it provides an important reference point.

This bit can be controversial, but it can be explained in terms of avoiding being locked into local maxima. When target state horizon is confined to the vicinity of the current state, one excludes significantly better but somewhat more distant possibilities:

Local Maxima diagram by Joshua Porter

What’s interesting is that typical corporate planning & funding horizons, which are rarely longer than a year, can be a very real limiting factor in being able to “see” far enough.

Tech Debt measures bases on the true, long-term target state, allow one to compensate for short-termism of planning horizons. It is therefore responsibility of architecture to not be bound by scope or timelines of individual projects, but look further ahead and provide a coherent, long-term vision that is aligned to technology strategy.

Agile & Bridge Building

Sometimes there’s persistent confusion on the relationship between architecture and agile delivery.

If you’re building anything more complex than a website or a mobile app with a cloud backend for the front end (BFF), then you need to identify and decide upfront things that are going to be hard to change later. But this doesn’t mean you have to figure out every detail in advance. Architecture in agile is very much about figuring out and deciding the key things that are hard to change later – and these vary case by case.

A good analogy would be in building bridges: you have to decide what type of a bridge it will be, e.g. a suspension bridge:

Manhattan Bridge Construction, Wikimedia Commons»

Or a movable bascule bridge:

© Alex Fedorov, Wikimedia Commons

The type of the bridge you’re building has to be figured out in advance since you can’t easily change one into the other mid-construction. It is a function of core constraints and requirements – the span, the height of the ships you want to be able to pass underneath it, etc.

However, what colour to paint the bridge or how to decorate it is a very flexible choice – no need to debate it upfront as you can change it at any time later (well, ideally before you bought a million dollars’ worth of paint).

Distributed Systems: Orchestration, Invocation, Events

A very typical concern in distributed systems is the need to coordinate logic in different domains / bounded contexts. Let’s say we have two domains, A and B. When A performs a bit of logic ‘a’, B needs to perform corresponding logic ‘b’. There are generally three ways to accomplish this:

1. Create an orchestrator:

Pros:

• A and B remain simple and agnostic of each other.

Cons:

• Orchestrator absorbs the complexity of knowing about both A and B, interfacing with APIs exposed by A and B, implementing rollback and/or retry logic to mitigate failures. 
• If either A or B change, the Orchestrator has to accommodate those changes. 
• If we want to add another version of B (let’s call it B′) or add another participant C, we have to modify the Orchestrator to deal with all such changes. 
• Orchestrator is in danger of becoming a god service — one that has to know everything about everything and that has to stay in sync with changes throughout the system. 
• This is a path to a distributed monolith.

2. Direct invocation:

Pros:

• Simplicity.

Cons:

• Tight coupling between A and B: A now has to understand the logic of B and interface with its APIs. If B changes, A might have to change. 
• This effectively forces A to be the orchestrator, violating the single responsibility principle. 
• A becomes stateful, holding state until the invocation of B successfully completes or handling failures. 
• While simple with just A and B, in the context of dozens or hundreds of services, point to point invocations quickly become difficult to manage, highly dependent on discovery and topology, etc. 

3.1. Event-Driven Architecture (Uncompensated): 

Pros: 

• A and B completely agnostic of each other 
• Loose coupling, allowing adding new publishers and subscribers without having to modify existing actors 
• With event sourcing based on appropriate message transport (e.g. Kafka), ability to replay/re-process events 

Cons: 

• Assumes reliability of message transport or ability by subscribers to compensate 

3.2. Event-Driven Architecture (with Observers): 

Pros: 

• A and B completely agnostic of each other 
• Loose coupling, allowing adding new publishers and subscribers without having to modify existing actors 
• With event sourcing based on appropriate message transport (e.g. Kafka), ability to replay/re-process events 
• Observer processes can be simple, only looking to match “done ‘a’” with “done ‘b’” without knowledge of details of ‘a’ and ‘b’, and flagging missing correlates 

Cons: 

• Requires a bit of extra sophistication to implement 

Conclusion:

Option 3.2 is clearly the best pattern, even if it requires familiarity with implementation of this pattern. I’d argue that if a team or org are embarking on building distributed systems, it should invest in building the requisite expertise to do it right. 

Luckily, implementations of this core pattern are increasingly available in the open-source world, without the need to build such plumbing from scratch.

Hot Topic: Multi-Cloud

Large enterprises and regulated global institutions are increasingly tackling the subject of utilizing multiple cloud providers - AWS, Microsoft Azure, Google Cloud Platform. This is a complex area with many competing positions and interpretations.

Why Multi-Cloud?

As investment into cloud-native implementations is ramping up, many large enterprises are resolving to tackle the question of multi-cloud adoption. Key concerns typically noted:

1. Avoiding concentration risk - i.e. ability to redistribute workloads across providers, initially or rebalancing in the future, meeting prudential regulators’ concerns around the dependence of our systems on a single provider
2. Retaining ability to utilize unique innovations from competing providers without migration that is too costly
3. Retaining possibility of commercial leverage and arbitrage for commodity services
4. Avoiding single vendor lock-in

Notably, achieving higher levels of resilience is not typically among these, as utilizing multiple availability zones and regions can in many geographies achieve availability targets without multi-cloud.

Dimensions

The biggest challenge in multi-cloud is avoiding having to descend to the lowest common denominator (VMs or IaaS-only) and losing the benefits of higher-level PaaS services (cloud databases, object stores, functions, event queues, etc.)

Still, it's worth outlining different angles from which multi-cloud can be approached:

1. Code-level portability - avoiding having to rewrite substantial amount of business logic or data transformation code in order to enable it to run in alternate cloud provider infrastructure
2. IaaS portability - deployment and run concerns, from utilizing provider-independent images and container configurations to networking (e.g. BYOIP), load-balancing and block storage 
3. Control plane portability - the hardest of them all. Notably includes monitoring/APM, alerts and end-to-end tracing, security & access management

A Measured Approach

I believe that code-level portability is the sweet spot for multi-cloud approach for new services and applications.

IaaS-only portability, e.g. forklifting whole VMs into cloud and eschewing the use of native/PaaS services cloud providers offer is fine for moving legacy workloads off-premise without refactoring (if you must), or for packaged apps where one doesn’t maintain source code and therefore has no control over code portability. However, for custom-developed, new code applications this approach would significantly reduce the value of the cloud proposition.

Control plane portability is today still quite difficult. Some of it can certainly be accomplished by utilizing common toolsets - e.g. OpenAPM, ELK, etc. across providers, but truly seamless control plane across multiple cloud providers is still either too difficult or too costly to achieve. Security and access management models vary across providers with few exceptions, e.g. BYOK has potential to broker root keys across cloud providers.

Code-level portability should be achievable more readily: after all, most PaaS dependencies - object stores, DBaaS, event queues and even functions/FaaS are being increasingly commoditized. It's possible to focus on common API subsets that eschew individual vendor tie-ins yet retain most of the power those native PaaS services offer. After all, why should the business or data transformation logic know whether they're publishing an event to Kafka, AWS Kinesis, Azure Event Hubs or Google Pub/Sub?

Even if common API subsets prove elusive in some cases, it should be possible to isolate dependencies into vendor-specific libraries/adapters, which can be shared across engineering teams. Dependency-injection and vendor-specific configurations added within build pipelines are another viable approach.

Historical Parallels

This in some ways parallels the path our industry travelled with respect to databases - from fully embracing vendor-specific features by writing business logic in stored procedures (with many good arguments marshalled in its favour) to common SQL standard and DB abstractions such as ORM that made cross-DB migrations much more feasible.

I fully expect multi-cloud to be an increasingly active area in open-source and open-core world with focus on common API abstraction for commodity PaaS.

It’s time to build the bank of the future

Original article published on NAB Tech Blog

How can large retail banks tackle the monumental challenge of modernising core systems architecture without sacrificing their role as economic guardians?

When automatic teller machines (ATMs) started appearing in the late 1960s, there was a sense that we’d taken our first real step into the future of electronic banking.

As people discovered the freedom to access cash at all hours on demand and the convenience of card payments began to gradually replace cash, a golden era of mainframe computing helped banks manage the rising volume of financial transactions.

Jump forward to 2018 and most of our dollars are now dispensed digitally (through shopfronts real and virtual), as trips to the ATM rapidly become a thing of the past.

As banks, we’ve successfully evolved our systems and technology to meet customers’ fundamental needs, and they now feel very comfortable with the digital representation of money. But customer expectations have evolved far beyond these basics.

The next revolution

The truth is that many banks are only just waking up to the next revolution in their sector, and I predict that developments in coming years could well be just as disruptive as the rise of electronic banking.

Some drivers of this are evident in the rise of fintech start-ups developing innovative financial tools and services increasingly relying on real-time data. Of course, examples of cross-subsidization and regulatory arbitrage can be readily found as well in the fintech sphere.

Many fintechs do not attempt to replace the core services traditional banks provide, but augment their value proposition. These can only succeed, however, if bank platforms are open enough to allow API-based integration, event-based pub/sub data exchange and other modern approaches to creating flexible, extensible ecosystems.

The question is, how do system architects (who’ve focused so much of their work on the protection of customers’ financial assets) now set about opening these systems up to the wider world?

The scale of the task is significant.

The foundation of a lot of today’s banking architecture is built on a giant network of tightly coupled legacy systems, each purpose-built for very specific business needs, with layers of bespoke integration and security, risk management and control implementations embedded throughout them.

Many of those systems took years to evaluate and implement, but now the race is on to replace them with far more open, flexible and agile ecosystems — all while retaining the robustness and security needed to beat ever-more-sophisticated financial crime and fraud.

Defending our customers’ assets is a massive job in itself — and not one we can afford to fail. So, how do we transform our banking systems without sacrificing our role as economic guardians?

A robust, coherent and ambitious architectural vision is crucial in the transformation.

Thinking big

Modern architectural approaches are not new but sometimes take surprising time to become well-established within our industry as best practices. Services replacing monolithic applications, improving loose coupling via event pub/sub and avoiding distributed monoliths, employing eventual consistency where appropriate to improve availability and resilience to partial failures, reactive patterns, establishing coherent API ecosystems are hardly new but require experience to get right and avoid pitfalls. Increasingly, CQRS, event sourcing and real-time stream processing provide powerful ways to move the needle still further.

Systems, software and enterprise architecture should be about how internals of the systems work, translating into how resilient, performant and scalable they are, not just what functional features they provide. It’s very much about what’s under the hood. However, in many institutions the practice has been focused on buying and integrating technologies, systems and applications rather than designing and building them.

Starting with a coherent strategy and set of architectural principles aiming to maximize flexibility, maintain resilience and security, yet accommodate increasing velocity of change and easier experimentation the business demands at the edge, is key. These principles are translated into a concise set of patterns, technology choices and reference implementations providing reliable guardrails to individual delivery teams aiming to maximize their velocity.

We should evaluate all systems — existing or new — according to these key principles and patterns, so none becomes an obstacle in achieving a flexible technology ecosystem.

Anything blocking or impeding movement toward our agreed target states definitionally becomes technical debt, to be managed proactively. Applied consistently, it also gives us a model to phase out legacy estates where it’s an obstacle to our goals.

People power

As is true for any organization — whether a startup or a large firm — the people we choose to drive our technological journey are as important as the architecture that supports it.

Financial services are experiencing a truly seismic shift, as we seek to solve some fundamentally complex problems with innovative ideas and best-of-breed tools.

And technology as a field is continuing to undergo rapid evolution, offering us new tools and choices, whether it be stream processing underpinning the transition from Big Data to Fast Data, machine learning utilizing GPU compute, or quantum computing promising solutions to previously computationally intractable problems at a price of upsetting the status quo in cryptography. It is a great time to be a practitioner in our field.

Distilling the right set of architectural principles and patterns and picking our technology building blocks wisely, tying these together into a coherent architectural vision in our complex, fast-evolving ecosystem — where failure simply isn’t an option — represents an exciting career challenge indeed.

If you’re interested in joining us, you can learn more about working in technology at NAB here.

On Well-Designed Event Models

I propose that events composing a well-designed event model should be of a form:

Where:

• E_Ai^x→y is the event communicating transition of aggregate/entity A_i  from business state S^x to business state S^y

• A_i is the i-th instance of aggregate or entity A consisting of attributes {α, β... ω} 

• State transition (S^x → S^y) is accompanied by changes in a subset of attributes of A_i {α, β... μ} to new values {α', β'... μ'}, these new values (if required, with previous values {α, β... μ}) are included in the data payload of the event E_Ai^x→y{α', β'... μ'}

This approach is in contrast with a trivial event model where events are simply attribute-level changes, e.g. instead of emitting E_Ai^x→y{α', β'... μ'}, emit the following series:

Ergo, developing a business state-centric event model requires starting with business state transition diagram for entity/aggregate A:

The benefit of business state-centric event model is to make explicit the state transitions (which typically involve changes to more than one attribute), instead of those having to be inferred/reconstructed by the consumer of attribute level events E^α_Ai {α'} = A_i {α'}.

Rationality of Rules vs. Rationality of Ends

"The problem of bureaucracy is that it values the rationality of rules over the rationality of ends."

– Matthew Taylor (with echoes of Max Weber) 

This can serve as a nice litmus test for when technology governance is in danger of devolving into a bureaucracy. This is why it’s so important to focus on tangible value in areas such as governance and technology architecture.

Microservices Architecture - Finding The Sweet Spot

Microservices architecture has been more in the spotlight lately, and with good reason.

As I wrote before in  Loose coupling and integration - the pendulum swings, splitting execution into multiple processes requiring remote invocation isn't free however.

In my recent discussions with many practitioners, I noticed that microservices architecture is typically put forth in opposition to a more coarse-grained approach where multiple domains  are lumped together, sometimes resulting in a 'God service' or 'everything service' anti-pattern.

However, it's dangerous to posit that more services is always better and create incentives to which teams and managers might respond - e.g. how many services did your team create last quarter? Nice, but that other team created twice as many!

So how to find the sweet spot in this spectrum? My answer is not to lump conceptually unrelated domains together. Another rule of thumb would be to look at persistence domains - if data/persistence architecture segregates domains properly, they should be exposed via separate service interfaces.

Fundamentally, services are simply public interfaces for domains. Each domain should have at least one. But creating too much fragmentation within each domain, exposing separate services for interrelated concerns shifts burden onto clients in terms of discovery, orchestration and invocation latency.

Ultimately, the goal of service interface design should be to allow clients to reason about the domain the service represents and integrate with it as simply and efficiently as possible.

Martin Fowler provides an  excellent write-up on microservices here.

Emergent software design?

Software seems to be the only area where we often try to combine designing a system with building it. Or declare that one should just start building it and the design will emerge.

Perhaps this is because in the material world, limitations of physics, materials properties and manufacturing processes are intuitively understood by non-specialists.

Building a house is a good example: a house is a composite of multiple subsystems: structural, electrical, plumbing, HVAC and so on. These subsystems won't just evolve independently to end up well-enmeshed in a coherent whole, and no future homeowner would assume such a thing and tell their contractors to just start building the house and not worry about blueprints.

Yet software has very real limitations in terms of complexity, physical resource utilization efficiency and so on, but these are not intuitively obvious to non-technologists. It is very easy to build something that appears functionally correct but is crippled in terms of scalability, maintainability, scalability, stability and so on.

So while not every detail has to be figured out and specified in advance, overall structure/architecture and how different subsystems should interact does need some forethought and definition.

Code maintenance & oil changes

Efficient organizations are typically KPI/metric-driven to a significant degree. Technology investments have to be translatable into business benefits, typically within 1 year horizon.

However, all code rots, with cruft and technical debt gradually accumulating within the codebase, necessitating refactoring and other maintenance that is bound to consume time and scarce resources. How can one demonstrate the necessity of such investment against competing priorities?

An analogy that comes to mind is engine oil changes. We spend time and money on them even though there's no direct, near-term proof that they are necessary. We essentially believe in their necessity because the experts tell us so. If we wait until the car starts showing signs of trouble, then it is probably too late and the repair will be far more expensive than an oil change.

So while managing technical debt is important, it is difficult to attach metrics to it that reliably demonstrate near-term benefits to business. Allocation of sufficient resources for it is then essentially a matter of trust between business and technology organizations.

Web apps - back to client-server?

Are we moving back to client server architecture for web applications?

The web started as a medium for sharing and linking content, with very limited interactivity. In time, ubiquity, elimination of software deployment to and maintenance on the clients, ability to update software on the server side transparently, SaaS and other factors made the web an application delivery channel as much as a content medium.

The biggest limitation that soon emerged is the quality of experience when using server-side UI state management - due to communication latency, responsiveness of UI leaves much to be desired, even as we moved from full page refreshes to partial updates, AJAX, parallelization of requests, etc.

Now that JavaScript has become a powerful, fully featured language, are we effectively moving back to client server model, in with rich JavaScript clients are running in browser containers and maintaining UI state on the client, with server side basically providing data and business logic processing?

The answer could well be yes, but to really match native apps in terms of quality of UI/experience, we need better resource management and control of the 'active' vs 'inactive' apps in the browser, when there are many windows & tabs open, refreshing, etc.

I think web as the medium for application delivery and browser as container is a trend that will continue and in time will force improvements in browsers, comm protocols and markup to enable better UI/experience.

Hiring Solution Architects!

Like many platform companies, PayPal has many strategic partners. Important partners come with their own requirements, often expressed as desired tweaks to core logic in multiple domains.

Needless to say, directly, programmatically implementing such tweaks across multiple domains would result in brittle, poorly maintainable code. The challenge is to ensure that there's enough flexibility in the key domains to support plurality of partner requirements without creating cruft/adding technical debt to the overall system.

This is best accomplished by technical solution architects on one hand working with partners to help shape their requirements against the fundamental rules and assumptions of the platform, and on the other hand, working with core domains comprising this platform to ensure enough flexibility is built in to support these requirements.

I'm currently looking for highly qualified individuals to fill the role of solution architects. Since all comments on this blog are moderated, you can apply by commenting on this post (I will not make submissions public).

Why is PayPal interesting technology-wise?

I work at PayPal and find that the set of problems we have to solve here is both quite interesting and challenging. How come?

From the outside, PayPal seems simple - a multi-channel, global payment platform.

Consider this, however - every government in the world forcefully imposes and continuously tweaks the rules governing money movements.

Typically, financial institutions have separate operating units in every market, with independent software/systems implementing those rules as well as market-specific product features. Sometimes, you even find major financial institutions with entirely separate systems serving different regions within US - so much so that in at least one instance an account registered on the East Coast could not be accessible under the same set of online credentials as accounts established on the West Coast.

Not so for PayPal: As a true Internet company, PayPal relies on a single code base and (distributed) infrastructure that serves all customers around the world. The benefit to customers is that you can instantly pay anyone globally. The flip side is a highly complex code base that has to support constant and rapid evolution and significant requirements flux.

This presents an interesting problem set in the areas of software engineering & architecture, otherwise rarely found on such a scale. And because of its phenomenal growth, PayPal has the wherewithal to tackle these problems and aggressively pursue innovation.

Pivoting towards rules-based application logic

Many applications are initially simple and straightforward, but gradually become more and more complex once specialization based on geographical market, user segment, etc. is required. The same problems arise when SaaS apps transition to true multi-tenant implementation from a set of bespoke, tweaked solutions for anchor clients.

At some point, managing programmatically-implemented use cases through complex, branching conditionals becomes too onerous. At this point, it's common to see the architeture pivot towards declarative, rules-based implementations.

The key to making this transition successful is in the interface between the app and the rules-based framework. Since these rules subsume portions of the app logic, one often has to pass a fairly rich application context as argument at the interface between them.

Constructing the context that is rich enough to maximize the diversity of effectively executable rules yet still well-structured is a good exercise in abstracting key inputs to app logic from mere artifacts of app's internal state.

What rules-based framework returns back to the app is effectively a set of decisions expressed as a set of parameters that define the behavior of the remaining app logic.

One final thought: mature enterprise apps often have multiple sub-domains implementing rules-based logic. In such cases, providing a façade that hides orchestration over all such sub-domains simplifies the app even more. However, this requires an even more careful modeling of both the context as well as result set.

Technical architecture - what's on your mind?

Questions I typically ask myself when designing a system:

1. Isolation/decomposition - what are the separable concerns?
    
2. Encapsulation - how to optimally package those concerns?
    
3. Intermediation & integration - how to compose and orchestrate over multiple components? Synchronous/blocking? Asynchronous/even-driven? Asynchronous/batch-driven?
    
4. Consistency/Availability/Partition-tolerance trade-offs? Eventual consistency? Does 'OK' mean 'fully committed transaction' or 'i've recorded enough information to achieve globally consistent commit at some point in the future'?
    
5. Business logic - programmatic, data-driven & rules-based?
    
6. Entity models - extensibility/flexibility vs. performance optimization?

These are rather generic, off the top of my head. 

Loose coupling & integration - the pendulum swings

We've all learned that separation of concerns is a fundamental good in software/systems design. Often, what is seen as a logical next step is to package these concerns into separately deployable artifacts - as services requiring remote invocation, for example.

Many a startup goes through these phases - start simply, with monolithic codebase and relational DB as a universal persistence mechanism. If the startup is successful, its growth inevitably requires more - more features, more specialization based on market/locale and user segments, more experimentation, integration of new and acquired capabilities, external systems, partners, etc.

At some point, the monolithic code base becomes too complex and the engineering organization too large for every engineer to understand most of the code. So the order of the day is... isolation! Draw reasonable boundaries within the code and attempt to create good interfaces between isolatable portions (hopefully hiding implementation details of each from the others).

And how can one isolate various code domains from each other in a way that makes enforcement of dependency management simple? Well - SOA (service-oriented architecture) of course!

So soon enough, instead of a monolithic code base, there's a rich panoply of services. But not without a cost - creating a lot of services and forcing remote invocation in the name of isolation is far more expensive than running execution within the memory space of a single process. Intermediation between a multitude of services becomes the next challenge - with far from trivial issues of orchestration, latency, topology & discovery, geo-distributed failover, etc. If processing of the same request now requires a cascade of a dozen or more service calls, how does one still make it performant? How to handle versioning required across common interfaces - supporting both the new and the legacy clients?

Attacking these issues by sheer brute force is usually an expensive proposition - with costs rapidly rising. The only answer I found is ensuring an adequate investment in quality technical design/architecture.

Processing batch/bulk requests

It's fairly common to get requests for creating a batch version of a previously transactional API. What are the main questions to ask when considering batch implementation:

1. Atomicity - what operations should be considered atomic, all-or-nothing (i.e. not admitting partial failures)?
    
2. Since sizable batches cannot be processed atomically, how to handle partial failures? This quickly leads to:
    
3. Idempotency - how to prevent erroneously submitted duplicate requests from creating snowballing failures and data corruption throughout the system?
    
4. Downstream effects - how to ensure that downstream systems that depend on asynchronous processes, such as ETL, work well with different load patterns created by upstream batch requests?

So introducing batch requests into the system without compromising consistency and while maintaining load & performance SLAs is not always a trivial task, which makes it interesting!

Agile vs. waterfall [software development] - the grand bargain

In waterfall-style development, the implied bargain is: product owner provides complete requirements and developer provides cost estimates and execution timeline. Of course, completeness/finality of requirements is pure fiction. So the bane of waterfall model is that every significant change in requirements requires re-estimation and re-planning, wasting huge amounts of time for all involved.

In agile, the core bargain is different - everyone saves time by not producing or parsing huge requirements docs (which often really are comprised of vast stretches of boilerplate with useful nuggets of info hidden here and there). Product owner gets flexibility for requirements and scope changes along the way, to a degree. Developer gets a committment to fund the implementation effort through initial launch and subsequent tweaks/experimentation.

Agile is not an excuse for lack of technical design - the architect or engineering lead still need to isolate the salient aspects of the product to ensure what they're building will at its core be ble to stand the test of time.

Against ivory towers

Does taking an architecture role relegate one to an ivory tower of high-level abstractions, processes and governance? Absolutely not!

Architects are responsible for ensuring there's no gap between high-level designs they create and what is ultimately implemented by developers. If an architect articulates those designs in high-level pattern language that is not readily consumable by the engineers, it's not good enough.

Architects have to provide clear translation of what those patterns mean in terms of CODE - be it draft implementation of core abstractions and key interfaces, pseudo-code explaining the patter, etc. But 'naked' high-level design documents are usually insufficient.

The art of abstraction

Technical architecture is in many ways the art of finding the right abstractions. This is why, beyond very rough guidance such as 4+1 view, I don't see architecture easily amenable to templates. Depending of what you're designing, the right set of abstractions will be different.