Chapter 1: Introduction

How can we engineer software systems that continues to function correctly when someone is deliberately trying to make it fail?

Answering this question requires proactive thinking; it needs us to examine the decisions made throughout the software lifecycle: what the system should do, what is should not do, how it is structured, how it is implemented, how it is tested, and how it changes after deployment.

This chapter introduces the necessary terminology used in the rest of this book.

From Programming to Software Engineering¶

Before we can talk about secure software engineering, let’s first define the last part of this term: what is software engineering after all?

You are likely very well familiar with the concept of programming, which is the act of writing code to solve a problem (e.g., writing the code for an app to help track monthly expenses). Software engineering is a broader discipline that encompasses programming but also includes the systematic approach to developing, operating, maintaining, and evolving software Sommerville (2022).

Writing code is definitely a key part of this process, but not the only thing! Real software system also needs us to think and define requirements, software architecture, configuration, data, dependencies, tests, documentation, deployment infrastructure, and the process (practices) followed to keep the system working over time.

A typical software development lifecycle includes:

1. Requirements: Determine what the system must do and what constraints it must satisfy.

2. Design: Decide how responsibilities, data, components, and interfaces will be organized.

3. Implementation: Translate the design into code and configuration.

4. Verification and testing: Evaluate whether the system behaves as intended and identify defects.

5. Deployment and operation: Release the system, monitor it, and respond to operational events.

6. Maintenance and evolution: Correct defects, adapt to new environments, and revise capabilities.

In civil engineering, a similar lifecycle might include surveying, planning, designing, constructing, inspecting, and maintaining a construction (e.g. a house). This would be a reasonable analogy, but software is more malleable than concrete. A house is built once and then maintained; software is built, maintained, and rebuilt many times. Thus, although this list appears sequential, development is usually iterative. A test result may reveal a misunderstood requirement. Operational data may motivate a redesign. A new dependency may change the system’s attack surface. The lifecycle is better understood as a set of connected, recurring activities than as a one-way assembly line.

Software does not wear out like hardware does, but it ages¶

Unlike hardware products, software products do not wear out. Instead, deployed software often appears to deteriorate. Its environment changes: dependencies are updated, attackers discover new techniques, assumptions become invalid, and repeated modifications make the system harder to understand. The software has not physically aged, but the relationship between the software and its environment has changed.

This distinction has an important security consequence. A system that was considered secure when released is not guaranteed to remain secure. New vulnerabilities may be discovered in its code or dependencies, features may introduce unsafe interactions, and attackers may find ways to violate assumptions that once seemed reasonable. Secure software must be maintained as deliberately as it is created.

Complexity is the central challenge¶

Software systems are extremely complex. Even a modest application can contain many components, third-party libraries, services, identities, permissions, data flows, and configuration options. Security failures often occur at boundaries:

• between components that make different assumptions;

• between trusted and untrusted data;

• between the permissions a user needs and the permissions the user receives;

• between an application and a third-party dependency; or

• between the system the designers imagined and the system that was deployed.

Software engineering gives us methods for managing complexity. Software security extends those methods to situations in which mistakes are not merely encountered by accident but actively searched for and exploited.

What Is Software Security?¶

Gary McGraw defines software security as engineering software so that it continues to function correctly under malicious attack McGraw (2004). This definition emphasizes three ideas.

1. security is a property of the software system, not only of the network or machine on which it runs. Firewalls, antivirus software, and intrusion-detection systems provide valuable layers of defense, but they cannot compensate for every insecure design or implementation decision.

2. security is an engineering concern. It must be considered in requirements, design, implementation, testing, deployment, and maintenance. A review immediately before release may find some defects, but fundamental design decisions can be expensive to change by then.

3. the system operates in an adversarial environment. Reliability asks whether the system behaves correctly when components fail or users make mistakes. Security also asks what happens when an intelligent opponent intentionally searches for inputs, actions, or states that developers did not anticipate.

Defense in depth¶

No single control can address every failure mode. Defense in depth uses multiple, mutually reinforcing layers so that the failure of one control does not immediately compromise the system. A web application might combine input validation, parameterized database queries, least-privilege credentials, network segmentation, monitoring, and reliable backups.

Some controls reduce the chance that a vulnerability is introduced. Some make exploitation harder. Some limit damage after compromise. Others help detect an attack or restore service. Their value comes from both their individual functions and their independence: several copies of the same flawed control do not create meaningful depth.

Building Security In¶

Software security aims to bring security into engineering from the beginning. This approach is often called building security in or shifting security left. The word left refers to diagrams that place early lifecycle activities, such as requirements and design, on the left and deployment on the right.

A proactive approach addresses security before a vulnerability reaches production. Examples include identifying security requirements, modeling threats, choosing architectures that isolate sensitive assets, adopting secure implementation patterns, reviewing code and dependencies, testing misuse cases, and planning for recovery.

A reactive approach addresses a vulnerability after it is discovered in a deployed system. Reactive work remains essential: no process eliminates every defect, and organizations must be able to detect, patch, contain, and learn from incidents. The mistake is to rely on reaction as the primary strategy.

The strongest programs do both. They prevent what they can, detect what remains, respond effectively, and feed lessons from incidents back into future engineering decisions.

Why timing matters¶

A missing authorization requirement found during design may be corrected by revising a model and a few interfaces. The same omission discovered after deployment may require changes to APIs, databases, clients, tests, documentation, and operational procedures—all while users remain exposed. Security incidents can also interrupt operations, trigger legal obligations, harm customers, and damage trust. Early work cannot guarantee a defect-free system, but it creates more opportunities to find consequential problems before attackers do.

Vulnerabilities, Bugs, and Design Flaws¶

A software vulnerability is a weakness that can lead to a security consequence. It exists when a defect or unsafe condition makes it possible to violate a security goal. The underlying defect may be in implementation or design.

Bugs¶

A bug is an implementation error. The intended design may be sound, but the code does not correctly realize it. Examples include an incorrect type conversion, an out-of-bounds memory access, an omitted authorization check, or improper handling of untrusted input.

Consider a service whose design requires it to verify a user’s permission before returning a record. If one endpoint accidentally omits the check, the implementation violates the design. Correcting the code brings the implementation back into alignment with the intended policy.

Design flaws¶

A design flaw is a fundamental problem in the system’s architecture, security mechanisms, or assumptions. The implementation may perfectly match the design, yet the system remains insecure.

Suppose an application performs authentication entirely on a client device and tells the server that authentication succeeded. Even flawless client code cannot make this design secure because an attacker controls messages sent to the server. The server must independently verify the identity or trustworthy proof of authentication. The allocation of trust and responsibility—not merely one line of code—must change.

The distinction matters because different techniques find different defects. Code review and static analysis may reveal implementation mistakes, while architecture analysis and threat modeling are better suited to flawed trust assumptions and missing security mechanisms. Secure development must address both.

Threats and Exploits¶

The terms vulnerability, exploit, and threat are related but not interchangeable.

• A vulnerability is a weakness that can lead to a security consequence.

• An exploit is software, data, or a sequence of actions that takes advantage of a vulnerability to cause unintended behavior.

• A threat actor is a person, group, organization, or automated agent capable of causing harm.

• A threat may refer to a potential cause of harm or, in some contexts, a class of harmful actions such as spoofing.

One vulnerability may have multiple exploits. An exploit may be performed manually or automated at scale. The relationship can be summarized as follows:

A threat actor uses an exploit to take advantage of a vulnerability and violate a security goal.

Imagine a file-processing service with a memory-safety vulnerability. A malicious file crafted to trigger the defect is an exploit input. The person sending it is the threat actor. If exploitation alters records, it violates integrity; if the service crashes, it violates availability; if memory is exposed, it may violate confidentiality.

Match the protection to the problem¶

• Protections against exploits block, detect, or contain attack techniques. Examples include endpoint protection, intrusion detection, application firewalls, exploit mitigations, and sandboxing.

• Protections against threat actors deter, identify, constrain, or respond to adversaries. Examples include authentication, monitoring, forensics, legal deterrence, and incident response.

• Protections against vulnerabilities prevent or remove weaknesses in the software. Examples include secure design, defensive programming, code review, security testing, and remediation.

Blocking a known exploit does not remove the vulnerability; correcting one vulnerability does not eliminate the threat actor; and identifying an attacker does not automatically restore a compromised system. Defense in depth depends on understanding which problem each control solves.

Security Goals¶

We cannot decide whether a system is secure without stating what must be protected. Security goals turn “make it secure” into properties that guide requirements, design, and tests.

The CIA triad¶

The three classic goals are confidentiality, integrity, and availability—the CIA triad.

Confidentiality means that information is disclosed only as intended. Encryption can support confidentiality, but so can access control, data minimization, and preventing secrets from appearing in logs or errors. A payroll system violates confidentiality if one employee can read another’s salary without authorization.

Integrity means that programs and data are protected against improper modification or destruction. A system may also need to identify who made an authorized change and preserve the order of transactions. A banking system violates integrity if an attacker can alter a balance or replay a transfer.

Availability means that the system and its information are accessible and operational when required. Redundancy, capacity planning, graceful degradation, rate limiting, backups, and disaster recovery can support availability. A hospital system that protects every record from disclosure but cannot retrieve one during an emergency is not secure in the way its users need.

These goals can create tradeoffs. Extensive verification may reduce performance and affect availability. Replication may improve availability while increasing the places where confidentiality must be protected. Good engineering makes these tensions explicit.

Additional goals¶

Identification and authentication establish and verify a claimed identity for a user, process, or device. Identification asks, “Who are you claiming to be?” Authentication asks, “What evidence supports that claim?”

Authorization determines what an authenticated entity may do. Proving that a user is Alice does not establish that Alice may approve a payment, read a medical record, or administer the system.

Accountability means that security-relevant actions can be traced to the responsible actor. Logs, audit trails, trustworthy timestamps, and protected identities can support it. Non-repudiation is a related but stronger property: it provides evidence that makes it difficult for an actor to credibly deny an action.

Privacy concerns a person’s ability to understand and control how information about them is collected, used, shared, retained, and deleted. Privacy and confidentiality overlap but are not synonymous. An organization may keep personal data confidential while collecting it without a valid purpose or using it in unexpected ways.

Security goals are contextual¶

The importance of each goal depends on the system. Availability may dominate for emergency communications, while confidentiality may dominate for a password manager. Security requirements should therefore name the asset, desired property, relevant conditions, and—where possible—a measurable criterion. “The system must be secure” is not testable. “Only clinicians assigned to a patient’s care may view the active treatment record, and every access must be recorded in a tamper-evident log” provides engineering guidance.

The Three Pillars of Software Security¶

At this point, we have established what software security aims to achieve and introduced the terminology needed to talk about security problems. But how do we put all of this into practice?

McGraw organizes software security around three pillars: knowledge, risk management, and touchpoints McGraw (2004). I like to think of these pillars as three questions:

1. Knowledge: What do we already know about software security problems and solutions?

2. Risk management: Which problems matter the most for this system?

3. Touchpoints: Where and how should we apply security practices throughout the software lifecycle?

None of these pillars is sufficient by itself. Knowledge without risk management can lead a team to spend time fixing issues that have little consequence for its system. Risk management without technical knowledge can overlook well-known attack patterns. And both are of limited use if they are not connected to concrete engineering activities.

Pillar 1: Knowledge¶

Software security has been around for long enough that we do not need to rediscover every problem ourselves. As the saying goes, we can see further by “standing on the shoulders of giants.” In our context, this means learning from vulnerabilities, attacks, design failures, secure design principles, attack patterns, and mitigations that have already been documented.

You have probably heard names such as Heartbleed, Shellshock, or Log4Shell. These names refer to specific vulnerabilities that affected real software. Studying them helps us understand not only what went wrong in one product, but also the more general classes of mistakes that may appear in other systems.

Several public knowledge catalogs help us do this systematically:

• The Common Weakness Enumeration (CWE) catalogs recurring types of software weaknesses, such as improper input validation or SQL injection.

• The Common Vulnerabilities and Exposures (CVE) program assigns identifiers to publicly disclosed instances of vulnerabilities in particular products or versions.

• The National Vulnerability Database (NVD) enriches CVE records with information such as affected products, related weaknesses, references, and severity metrics.

• The Common Vulnerability Scoring System (CVSS) provides a standardized way to describe and score the technical severity of a vulnerability.

The difference between a weakness and a vulnerability is important. A weakness is a general type of mistake; a vulnerability is an actual instance of a security problem in a real system. If you are familiar with object-oriented programming, you can think of a weakness as a class and a vulnerability as an object instantiated from that class.

For example, CWE-89 describes improper neutralization of special elements used in an SQL command—what we usually call SQL injection. The well-known “Little Bobby Tables” example illustrates this weakness: an application constructs a query by concatenating an untrusted name directly into an SQL statement. A particular application containing that mistake has a vulnerability; CWE-89 names the broader weakness that helps us recognize and prevent similar vulnerabilities elsewhere.

Knowledge catalogs give developers, architects, testers, and security analysts a shared vocabulary. They also connect known problems to the artifacts and practices where we can address them. A weakness involving unsafe input handling may influence coding guidelines and code review, while a weakness involving unencrypted communication may point to an architectural problem.

Pillar 2: Risk Management¶

Knowing that a system could contain hundreds of different weaknesses does not tell us what to do first. Time, people, and money are finite. Risk management helps us identify, rank, track, and understand the security risks that matter to a particular system.

A common simplified model is:

Risk = probability × impact

For software security, we can make the same idea more concrete:

Security risk = probability of successful exploitation × value of the affected asset

An asset is anything that has value and therefore needs protection. Some assets are domain-specific, such as patient records in a healthcare system or flight-control commands in an aerospace system. Others appear in many domains, such as credentials, cryptographic keys, personal data, computing resources, and the availability of a service. Assets do not have to be tangible. Trust, reputation, privacy, and the integrity of a decision can also be valuable.

The probability of exploitation depends on several factors. Does a relevant vulnerability exist? Is the vulnerable component reachable by an attacker? Is an exploit publicly available? How difficult is it to use? Does the system attract targeted attacks? What controls already reduce the attacker’s opportunity or limit the damage?

The impact depends on what the attacker can reach and which security goals may be violated. The same implementation bug may be low risk in an isolated demonstration program and critical in an internet-facing service that processes sensitive data. This is exactly why severity and risk should not be treated as synonyms.

Risk management is not something we should postpone until testing. Assets, trust boundaries, and high-impact failures can—and should—be identified during requirements and design. Doing so helps the team decide where stronger controls, deeper analysis, or additional testing are justified.

Pillar 3: Software Security Touchpoints¶

The third pillar turns knowledge and risk decisions into engineering work. Software security touchpoints are security best practices applied to specific artifacts at different points in the software lifecycle. Some are constructive: they help us build security into requirements, designs, code, and operations. Others are destructive: they deliberately challenge the software to find ways it can fail.

The seven touchpoints are not seven isolated phases, nor are they a checklist that must be performed exactly once. They overlap, inform one another, and should be applied according to risk.

Code review¶

In a code review, one or more people—or automated tools working alongside them—inspect source code for defects. A security-focused review asks questions such as: Does untrusted input reach a sensitive operation? Is authorization checked on every relevant path? Are secrets handled safely? Can an error leave the system in an unsafe state?

Code review is particularly useful for finding actual vulnerabilities in implementation. But remember our earlier distinction: carefully reviewing every line of code will not necessarily reveal that the system is enforcing the wrong security model. That is why we also need architectural analysis.

Architectural risk analysis¶

Architectural risk analysis examines designs and specifications to find flaws before they become deeply embedded in code. It considers assets, components, data flows, trust boundaries, attack surfaces, and the assumptions made by the design.

Imagine a satellite control system that transmits sensitive commands between components without encryption. Every component might be correctly implemented according to its specification, yet the architecture exposes the commands to disclosure or modification. This is a design flaw, and correcting it may require changes to protocols, key management, interfaces, and operational procedures.

Penetration testing¶

Penetration testing examines a functioning system in its deployed—or realistically simulated—environment from an attacker’s perspective. Testers gather information, look for reachable weaknesses, attempt exploitation within an authorized scope, and report evidence and recommendations.

Penetration testing is valuable because it evaluates the integrated system rather than one artifact in isolation. However, it happens relatively late and can demonstrate only the vulnerabilities the testers manage to find. A successful penetration test is not proof that no other vulnerabilities exist.

Risk-based security testing¶

Risk-based security testing derives tests from known risks, security requirements, abuse cases, and design decisions. Unlike penetration testing, where the tester may begin with limited internal knowledge and behave like an attacker, risk-based testing often uses detailed knowledge of the design to verify specific security behavior.

Consider a heartbeat mechanism used to detect a failed component. A sender periodically transmits heartbeat messages, a receiver records their arrival, and a monitor is notified when messages stop. Risk-based tests should verify more than the happy path: Does the sender use the correct interval? What happens when it fails? Does the receiver recognize that no heartbeat arrived? Does it notify the monitor after the specified expiration time? By deriving tests from the design and its failure risks, we check whether the intended protection actually works.

Abuse and misuse cases¶

Traditional use cases describe what legitimate users want the system to do. Abuse cases (also called misuse cases) ask what an attacker, careless user, or unexpected environment might make the system do. In other words, they help us define what the software cannot and will not do.

This requires us to challenge our assumptions about user behavior. What if a user enters a string where we expected a name? What if an insider has valid access but misuses it? What if requests arrive in an unexpected order? What if someone intentionally searches for the boundary between two permission levels?

Abuse cases can be documented as anti-requirements: explicit statements of undesirable behavior that the system must prevent or mitigate. They are informed by relevant attack patterns and weaknesses, and they should consider both outsiders and insiders. For an online store, a normal use case may be “search for products”; a corresponding misuse case may be “inject SQL through the search field.” That misuse case can lead directly to an input-validation requirement and concrete security tests.

Security requirements¶

Security requirements translate security goals and risks into explicit, verifiable obligations for the system. A useful requirement needs more than a security-related word such as secure, protected, or strong. It should identify what must be protected, under which conditions, and how satisfaction can be evaluated.

When working with security requirements, pay attention to three things:

1. Definition: Is the security requirement explicitly stated?

2. Assumptions: What must be true about users, components, dependencies, or the environment for the requirement to work as intended?

3. Satisfaction: Does the requirement support the relevant security goal, and does the implemented system actually satisfy the requirement?

A requirement such as “user passwords shall expire” is incomplete unless the policy, rationale, exceptions, and expected behavior are clear. More importantly, we should ask whether password expiration is the right control for the identified risk rather than writing requirements simply because they sound security-related.

Security operations¶

Finally, security operations applies security to the deployed system and to the infrastructure used to build and run it. This includes secure configuration, secrets management, dependency and patch management, logging, monitoring, incident response, backups, and recovery. In a DevSecOps environment, these concerns are integrated into automated build, deployment, and operational workflows.

Operational artifacts are software too. Infrastructure-as-code scripts, container definitions, and deployment configurations can contain hard-coded secrets, weak cryptography, overly broad permissions, insecure network rules, or unsafe defaults. Building secure application code is not enough if the deployment process exposes it.

How the pillars work together¶

Suppose a team is building an online store. The knowledge pillar tells the team that SQL injection is a well-known weakness (CWE-89), explains how it occurs, and points to established mitigations. Risk management tells the team which database operations and assets deserve the most attention—for example, customer credentials, payment records, and order integrity. The touchpoints put that understanding into action: an abuse case considers malicious search input; a security requirement prohibits untrusted data from changing query structure; architectural analysis examines trust boundaries; code review checks query construction; risk-based tests exercise malicious inputs; penetration testing evaluates the deployed application; and security operations monitors suspicious behavior.

That is the central idea: software security knowledge tells us what to look for, risk management tells us what matters most, and touchpoints tell us where to act. Together, the three pillars turn “make the software secure” into an engineering process.

Putting the Concepts Together¶

Consider a web application that lets university students view grades.

• Asset: grade records and the service that provides them.

• Security goals: students may view only their own grades (confidentiality); unauthorized users may not alter grades (integrity); records must be available during defined hours (availability); and accesses and changes must be attributable to an account (accountability).

• Threat actor: a student attempting to view another student’s record.

• Vulnerability: the server accepts a student identifier from the browser but does not verify that it belongs to the authenticated account.

• Exploit: the student changes the identifier and retrieves another student’s record.

• Root cause: if the design assigns authorization to browser code, it contains a design flaw; if the server-side design requires authorization but one handler omits it, the defect is a bug.

• Proactive controls: define authorization requirements, centralize server-side access control, analyze trust boundaries, review the design, and test cross-account requests.

• Reactive controls: detect unusual access, investigate audit records, disable compromised accounts, correct the vulnerability, notify affected people when required, and learn from the incident.

• Knowledge: use catalogs such as CWE to identify relevant weakness classes and known mitigations.

• Risk management: prioritize unauthorized grade disclosure and modification according to their likelihood and impact on students and the university.

• Touchpoints: write misuse cases, define authorization requirements, analyze the trust boundary between browser and server, review access-control code, test cross-account requests, and monitor the deployed service.

Precise language matters because each term identifies a different part of the problem and suggests a different engineering response.

Chapter Summary¶

Software engineering manages the complexity of creating and evolving software systems. Software security applies that discipline in an adversarial setting, aiming to ensure that systems continue to satisfy their goals under attack.

Security must be built into the lifecycle and supported by multiple layers of defense. Proactive practices reduce the vulnerabilities that reach production; reactive capabilities detect, contain, and correct the failures that remain.

A vulnerability is a weakness with security consequences. It may arise from an implementation bug or a design flaw. An exploit takes advantage of a vulnerability, while a threat actor is a potential source of harm. Effective defenses address these distinct parts of the problem.

Security is meaningful only in relation to explicit goals. Confidentiality, integrity, and availability form a foundation, supplemented by identification, authentication, authorization, accountability, non-repudiation, and privacy.

The three pillars connect these ideas to practice. Knowledge catalogs help us learn from previously discovered weaknesses and vulnerabilities. Risk management helps us identify assets and prioritize work according to the probability and impact of exploitation. The seven touchpoints apply constructive and destructive security practices to requirements, designs, code, tests, deployed systems, and operations.

Review Questions¶

1. Why is software engineering broader than programming?

2. Why can an unchanged program become less secure over time?

3. What does it mean to shift security left? Why does it not eliminate the need for incident response?

4. Explain the difference between a bug and a design flaw. Give one example of each.

5. Distinguish a vulnerability, an exploit, and a threat actor.

6. Why might blocking a particular exploit leave a system vulnerable?

7. Describe confidentiality, integrity, and availability requirements for an online voting system.

8. How do authentication and authorization differ?

9. Give an example of a system that preserves confidentiality but violates privacy.

10. Choose a system you use regularly. Identify one asset, one threat actor, one plausible vulnerability, and two layers of defense.

11. Explain the difference between a CWE and a CVE. Why is the class-and-object analogy useful?

12. Why should a CVSS score not be treated as a complete measurement of organizational risk?

13. Name the three pillars of software security and explain how they support one another.

14. Contrast penetration testing with risk-based security testing.

15. Write one normal use case and one corresponding abuse case for an online banking system.

16. Match each of the seven touchpoints to its primary software artifact.