Skip to content
Search

System Architecture

lastmod 2025-08-19created 2014-01-118 edits

Subfields of system architecture

System Architecture is a framework that describes how a system works. It describes what each component of the system is, how they interact, and how information is exchanged in order to achieve the system's purpose. Definitions of system architecture vary, but taken together they share the following basic requirements.

  • It represents the system's composition and operating principles.
  • System components (parts) are described in enough detail to support design and implementation. (IEEE 1471 or TOGAF, etc.)
  • Relationships between components and with the system's external environment are depicted.
  • It considers the requirements specification and the entire lifecycle of the system.

Softwares

Good system design

Good system design is one that doesn't look complicated and runs for a long time without notable problems

  • Handling state is the hardest part of system design, and it is important to minimize the number of components that store state wherever possible
  • The database is mainly where state is kept, so an approach focused on schema design, indexing, and resolving bottlenecks is needed
  • Caching, event processing, background jobs, and the like should be introduced carefully for performance and maintainability, and overuse is best avoided
  • Rather than complex designs, appropriately using well-proven simple components and methodologies is key to building sustainable, stable systems

Definition of system design and the overall approach

  • If software design is assembling code, system design is the process of combining various services
  • The main components of system design are app servers, databases, caches, queues, event buses, proxies, etc.
  • Good design elicits reactions like "no particular problems", "it finished more easily than expected", and "you don't need to worry about this part"
  • Conversely, complex and conspicuous design can hide fundamental problems or indicate over-engineering
  • Rather than introducing a complex system from the start, it is better to evolve gradually from a minimal, working simple structure

Distinguishing state and stateless

  • The trickiest part of software design is state management
  • A service that returns results immediately without storing information (such as GitHub's PDF rendering) is stateless
  • By contrast, a service that writes to a database manages state
  • It is best to minimize stateful components in the system. This lowers the system's complexity and the likelihood of failures
  • A structure where only one service manages state, and the remaining services focus on stateless roles such as API calls or emitting events, is recommended

Database design and bottlenecks

Schema and index design

  • Storing data requires a human-readable schema design
  • An overly flexible schema (e.g. storing everything in a JSON column) can burden application code and performance
  • Appropriate indexes should be set up based on the columns that are frequently queried. Indexing everything actually creates needless overhead

How to resolve bottlenecks

  • Database access often becomes a heavy bottleneck
  • Processing complex data inside the database with JOINs and the like, rather than in the application, is generally better for performance
  • When using an ORM, beware of the mistake of issuing queries inside a loop
  • Splitting queries as needed to control database load or query complexity is another option
  • Distributing read queries to replicas (read-replica) to reduce load on the primary (Write) node is an effective strategy
  • When a large volume of queries arrives, transactions and write operations can easily overload the database, so query throttling (limiting) should be considered

Separating slow and fast operations

  • Operations users interact with need a response within a few hundred milliseconds
  • For long-running operations (e.g. large PDF conversions), a pattern of providing only the minimum immediately on the front end and handing the rest off to the background is effective
  • Background jobs typically run with a queue (e.g. Redis) paired with a job runner
  • For jobs scheduled far in the future, creating a separate DB table instead of Redis and running them with a scheduler is practical

Caching

  • Caching contributes to cost savings and better performance when identical or expensive operations are repeated
  • Junior engineers who have just learned about caching usually want to cache everything, while more experienced engineers are more careful about introducing caches
  • A cache introduces new state, so there are risks such as sync issues, errors, and stale data
  • It is advisable to first try performance improvements such as adding indexes to queries before applying caching
  • Large caches can also be stored periodically in a document store such as S3/Azure Blob Storage instead of Redis/Memcached

Event processing

  • Most companies have an event hub (e.g. Kafka), with various services doing distributed processing based on events
  • Rather than overusing events, a simple request–response API design is more useful for logging and troubleshooting
  • Event-based processing is suitable when the sender doesn't need to care about the receiver's behavior, or for high-volume, latency-tolerant scenarios

How data is delivered: push and pull

  • There are two ways to deliver data: Pull (response after request) and Push (automatic delivery on change)
  • The Pull approach is simple but leads to repeated requests/overload problems
  • The Push approach delivers data to clients immediately when it changes on the server, so it is efficient and good for keeping data up to date
  • Handling a large number of clients requires scaling the infrastructure (event queues, multiple cache servers, etc.) to fit each approach

Focus on Hot Paths

  • A hot path means the most important path in the system through which the most data flows
  • Hot paths have few options, and a design failure can cause serious problems for the entire service, so careful design is essential
  • Rather than minor features with many options, it is effective to focus on hot paths and allocate resources to their design and testing

Logging, metrics, tracing

  • To diagnose causes when failures occur, you should actively write detailed logs for the unhappy path
  • Collecting basic observability metrics such as system resources (CPU/memory), queue size, and request/job time is necessary
  • Instead of looking only at averages, you must also watch distribution metrics such as p95 and p99 latency. A small number of the slowest requests may be problems for your key users

Kill switches, retries, failure recovery

  • Strategic use of kill switches (temporarily blocking the system) and retries is important
  • Blind retries only burden other services; controlling requests in advance with a circuit breaker or similar is what makes them effective
  • Introducing an Idempotency Key makes it possible to prevent duplicate work when the same request is reprocessed
  • In some failure situations you need to choose between fail open and fail closed. For example, for Rate Limiting, fail open (allow) has less impact on users. Authentication, on the other hand, must fail closed

Wrap-up

  • Some topics such as service separation, containers, adopting VMs, and tracing were left out, but using well-proven components in the right places leads to the most stable systems in the long run
  • Technically special designs are actually very rare, and boringly simple designs are the ones used most often in practice
  • In essence, good system design is unremarkable: the process of safely combining well-proven methodologies

See also

Favorite site

author Zer0 sitemap