Shifting the focus from deploying systems to reconfiguring existing resources to increase enterprise capability.
Whether we recognize it or not, system architecture is evolving. For the past few decades, system architecture has been like building architecture: Decide what the system needs to do, identify the major subsystems that are needed and how they wi8ll connect, and continue decomposing until there is enough detail for development teams to build out each subsystem and integrate them to create the desired system. That patter has been changing in recent years, but the rate of change is increasing.
The impact of networks
System architecture practices began to change as networked systems became more common in the 1990s. The emergence of the client-server architecture as the dominant design pattern made it necessary to include network interfaces. Systems were no longer monoliths that were deployed in one place and used from a fixed set of terminals but were spread out over a potentially large geographic area.
The next major effect of networks on the evolution of system architecture was the desire to integrate systems. It did not take long to realize that entering the same data into different systems was time-consuming and error-prone, so we began to try and integrate systems so they could share data. This led to the development of service-oriented architecture (SOA). The basic idea of SOA was that capability providers would make their offerings available as services that could be invoked by any application on the network. A “service” is nothing more than a well-defined interface to some desired capability. SOA promised an era of dynamically composable applications that could be adapted to new business needs as they emerged, without the need to re-code applications.
The rise of services
Like many widely hyped new technologies, SOA never quite lived up to its initial promises. But also like many hyped technologies, a decade after the hype has died down, we are seeing real benefits of the SOA philosophy. Many capabilities are available as services, and they are being used by more enterprises. For example, Facebook, Google, and others all offer authentication services. If you run a web site and want to authenticate users before you allow them to access all features of the site, you do not need to host your own authentication subsystem—you can just use one of those offered as services. In a similar fashion, comment threads, social media integration, user statistics, and many other functions are also offered as services. The entire cloud computing revolution is really just the conversion of computing hardware to an on-demand service.
While it did not take the form originally envisioned, the SOA revolution most definitely happened. Most enterprise integration efforts today focus on making system interfaces publicly available. This is often referred to as the “application Programming interface (API)-first philosophy.” The most famous example of the API-first philosophy is probably the missive that has come to be known as “the Steve Yegge rant,” where he chides Google for not adopting the Amazon philosophy of API-first design. The basic thrust of the rant is that all capabilities should be exposed on the network through APIs to both promote integration and to minimize the amount of duplicative functionality the enterprise is producing (and paying for).
How APIs are driving system architecture
So far, the primary effect of any API-first mandate has been to make developers ensure they document their APIs and publicize them. But a major thrust of the Amazon API-first mandate was to reduce the costs incurred from developing duplicate capabilities in multiple systems. Because most enterprises do not update all their systems every few years, any API-first mandate will take time to show real effects in the enterprise. But over time, those effects will make themselves felt, especially when an API-first mandate is combined with a reuse-before-build mandate that requires system developers to reuse capabilities available in the enterprise before building new ones.
Toward capability-focused architecture
The enterprise ecosystem we are all facing today is one in which an ever-expanding set of functions are available as a service, particularly in a cloud environment. Cloud providers compete to offer increasing numbers of functions, and it is already possible to develop basic systems by stitching together a few services with some glueware or a scripting language. In doing so, developers can create a minimally functional system in a few weeks instead of a few months. This basic system can be rapidly improved by stitching in new services or installing off-the-shelf modules where services are not available. In such an environment, a months-long design phase that attempts to work out the details of the system before building begins does not make sense. We need a new way of thinking about system architecture and design.
In an enterprise that already has a number of services available, building a new system should begin with clearly defining what functions the intended system needs to perform and comparing that to a list of functions that are already available as services. This will reveal how much of the desired system is already available in the enterprise and how much needs to be built. The difference between functions already available and functions needed defines the capability gap between the enterprise’s current capabilities and the desired capabilities. As we move into the future, the primary task of the system architect will evolve from designing an entire system to defining the current capability delta and designing the best means for closing that gap.
We are not quite to the point where this kind of capability-focused architecture is easy. Our ability to understand what services are available across the enterprise is severely limited. Any honest network administrator will admit that they don’t really have a handle on the full list of services available on their network. They may know which machines are connected to the network, what software is running on each, and which ports and protocols are open on each machine. But that information only tells us about the network-level aspects of those things—it doesn’t reveal anything about how those things are being used. For example, a system on the network with port 8443 open and accepting HTTP connections may be offering up simple web pages or it may be offering a number of REST services through that interface.
There are means of overcoming this lack of understanding, but most of them are manual. For example, maintaining a wiki that lists available services in the enterprise requires that developers add the services they have deployed and maintain that listing. And automated means for identifying and cataloging service interfaces in near-real-time would be more efficient. But that’s a topic for another time.
There are exceptions
There are some areas where traditional system architecture will survive because of the environment where the system needs to operate. Any operating environment where reaching out to the full enterprise for a function is problematic will need to do full-scope system design the old-fashioned way. For example, aircraft flight control systems really cannot rely on the ability to call a service hosted on the ground for any function that relates to flight safety. In a similar vein, satellite systems and other types of embedded software will need to provide all critical functions locally.
