“Transactions guarantee that your work completes correctly. Isolation levels determine what everyone else is allowed to see while that work is happening.”

In the previous article, we explored database transactions and learned how they ensure multiple database operations either succeed together or fail together. Transactions protect applications from partial execution, preventing situations where money is deducted from one account without being deposited into another or where inventory is reduced without successfully creating an order.

But transactions solve only part of the problem.

Modern applications rarely have just one user interacting with the database at a time. Thousands of customers may be placing orders, updating records, making payments, or querying reports simultaneously. Each of these actions runs inside its own transaction, and more often than not, several transactions are accessing the same data at exactly the same time.

This raises an important question:

What should one transaction be allowed to see while another transaction is still running?

Imagine opening your banking application to check your account balance.

At the exact same moment, your employer’s payroll system is depositing your monthly salary into the same account.

Should your transaction see the new balance immediately?

Should it continue seeing the old balance until the salary transaction finishes?

Should it wait until the payroll transaction completes before showing you anything at all?

Each answer is technically valid depending on how the database is configured.

Now imagine an online store where only one laptop remains in stock.

Customer A begins placing an order.

Before their transaction finishes, Customer B checks the product page.

Should Customer B still see one laptop available?

Should they see zero?

Should they wait until Customer A’s purchase either succeeds or fails?

Again, the answer depends on the database’s isolation level.

Isolation levels define the rules governing how concurrent transactions interact with one another. They determine whether one transaction can observe another transaction’s work before it has been completed, whether repeated reads always return the same result, and whether new rows appearing during a transaction should be visible immediately.

Although isolation levels are often introduced as an advanced database topic, they’re really about one simple idea:

How much of another transaction’s work should your transaction be allowed to see?

The answer has significant consequences for both correctness and performance.

In this article, we’ll explore why isolation levels exist, the concurrency problems they solve, the four SQL standard isolation levels, and how to choose the right one for your application.

Why Isolation Exists

To understand isolation, imagine you’re reading a book in a library.

Halfway through chapter three, someone walks over, quietly replaces several pages with new ones, and walks away.

You continue reading without realizing anything changed.

The beginning of the chapter describes one story.

The ending describes another.

Nothing makes sense.

Databases can experience a remarkably similar problem.

When multiple transactions execute simultaneously, each transaction may be reading data while another transaction is actively changing it.

Without rules governing these interactions, applications could make decisions based on incomplete information, outdated values, or data that is eventually discarded.

Isolation exists to prevent these situations.

Rather than allowing every transaction unrestricted access to every change happening in the database, the database controls what each transaction can observe and when it can observe it.

Think of it as putting walls between transactions.

Some walls are very thin.

Transactions can see almost everything happening around them.

Other walls are much thicker.

Transactions operate almost as though they’re the only users of the database.

The thicker the wall, the more isolated the transaction becomes.

The Trade-Off Between Consistency and Performance

At first glance, it might seem obvious that every database should simply use the highest possible isolation level.

After all, if stronger isolation produces more consistent data, why wouldn’t every system choose it?

The answer lies in performance.

Imagine a supermarket with only one checkout counter.

Every customer waits patiently in line.

Because only one cashier is serving customers, inventory updates happen one at a time.

Mistakes are rare.

Unfortunately, the queue becomes enormous.

Now imagine opening ten checkout counters.

Customers move much faster.

However, all ten cashiers are now updating the same inventory system simultaneously.

Keeping everything synchronized becomes much more difficult.

Databases face exactly the same challenge.

Higher isolation levels provide stronger guarantees about data consistency, but they often require additional locking, coordination, and waiting.

Lower isolation levels allow more transactions to execute concurrently, increasing throughput and reducing latency, but they also increase the likelihood that transactions observe changing data.

Isolation levels are therefore a balancing act between two competing goals:

• Consistency, ensuring every transaction sees predictable and reliable data.
• Concurrency, allowing as many users as possible to interact with the system simultaneously.

Different applications make different choices.

A banking application processing financial transfers typically prioritizes correctness over raw performance.

An analytics dashboard generating sales reports might tolerate slightly older data if it means thousands of users can run reports simultaneously without slowing the system.

Neither approach is universally correct.

The appropriate isolation level depends entirely on your business requirements.

Concurrency Anomalies: The Problems Isolation Levels Exist to Solve

Isolation levels were not invented simply to make databases more complicated.

They exist because concurrent transactions can produce behaviors that most developers would consider surprising—or even dangerous.

These unexpected behaviors are collectively known as concurrency anomalies.

Every isolation level is essentially a trade-off between preventing these anomalies and maintaining good performance.

Before discussing the isolation levels themselves, it’s important to understand the problems they are designed to solve.

The four anomalies you’ll encounter most often are:

• Dirty Reads
• Non-Repeatable Reads
• Phantom Reads
• Lost Updates

Each represents a different way concurrent transactions can interfere with one another.

Let’s begin with the simplest.

Dirty Reads

Imagine Alice has KES 50,000 in her account.

She initiates a transfer of KES 20,000 to another account.

The banking system begins processing the transaction.

The first step deducts the money from Alice’s balance.

Before the transaction finishes, another process—perhaps an ATM balance inquiry or an online banking session—checks Alice’s account.

At that moment, it sees a balance of KES 30,000.

Everything seems perfectly normal.

Then something unexpected happens.

The transfer fails because the destination account no longer exists.

The database rolls back the transaction.

Alice’s balance immediately returns to KES 50,000.

The second transaction has now made a decision based on information that never officially existed.

It observed data that was eventually discarded.

This is known as a Dirty Read.

A dirty read occurs when one transaction reads data written by another transaction before that transaction has been committed.

The easiest way to understand it is to imagine reading someone’s unfinished draft before they’ve decided whether to keep or delete it.

The version you read may never become the final version.

Making business decisions based on that draft could lead to incorrect outcomes.

Fortunately, most modern relational databases prevent dirty reads by default because they are rarely desirable in business applications.

The SQL standard still defines them because they help explain the spectrum of isolation levels.

Timeline of a Dirty Read

Transaction A                     Transaction B

BEGIN

Balance = 50,000

↓

Update Balance = 30,000

                               Read Balance = 30,000 ❌

↓

Transfer Fails

↓

ROLLBACK

Balance returns to 50,000

Transaction B has observed a value that disappeared moments later.

From the perspective of the database, that balance never officially existed.

Yet another transaction already acted as though it did.

This is precisely the type of inconsistency isolation levels are designed to prevent.

Non-Repeatable Reads

Suppose you’re building an online banking application.

A customer opens the app and views their account balance. At that moment, the database reports a balance of KES 50,000.

The customer decides to transfer KES 40,000 to another account, but before confirming the transfer, the application performs one final balance check to ensure sufficient funds are still available.

This seems like a perfectly reasonable workflow.

However, between the first balance check and the second, another transaction deposits KES 100,000 into the same account.

When the application performs the second query, the balance is no longer KES 50,000.

It’s now KES 150,000.

Nothing is technically wrong.

The second transaction committed successfully.

The balance genuinely changed.

The surprising part is that the same transaction read the same row twice and received two different answers.

This phenomenon is known as a Non-Repeatable Read.

Unlike a dirty read, the second transaction isn’t reading uncommitted data. Every value it sees has been permanently committed to the database.

The inconsistency comes from the fact that another transaction modified the row while the first transaction was still running.

Imagine reading yesterday’s newspaper while someone keeps replacing pages with today’s edition.

The information isn’t incorrect.

It’s simply inconsistent because the document changed while you were reading it.

For many applications, this isn’t a problem.

If you’re refreshing a weather dashboard or checking the number of users currently online, it’s perfectly acceptable for values to change between two queries.

However, systems that rely on a stable snapshot of data—such as financial reporting, payroll processing, or end-of-day reconciliation—often require the same query to return the same result throughout the entire transaction.

Timeline of a Non-Repeatable Read

Transaction A                     Transaction B

BEGIN

Read Balance = 50,000

                               BEGIN

                               Deposit 100,000

                               COMMIT

Read Balance = 150,000 ❌

COMMIT

Transaction A never modified the balance itself.

It simply asked the same question twice and received two different answers because another committed transaction changed the underlying data in the meantime.

Real-World Example: Updating a Customer Profile

Consider an insurance application where a customer service representative opens a customer’s profile.

The representative spends several minutes reviewing the information before approving a policy update.

Meanwhile, another employee updates the customer’s phone number and address.

When the representative finally clicks Save, the application may now be working with information that is different from what was originally displayed.

Depending on how the application handles these changes, it could accidentally overwrite newer data or make decisions using outdated information.

This isn’t a database bug.

It’s simply the natural consequence of multiple users interacting with the same record at the same time.

Applications that require users to work with a consistent view of the data often use higher isolation levels or optimistic concurrency controls to detect these situations before committing changes.

Phantom Reads

Now let’s consider a different scenario.

Instead of reading a single row twice, imagine you’re querying an entire collection of rows.

Suppose you’re generating a report showing all loan applications submitted today.

Your first query returns:

Loan Applications Submitted Today

-------------------------------

Loan #101

Loan #102

Loan #103

Total: 3

While your report is still running, another loan application is submitted and committed to the database.

A few moments later, your transaction performs the exact same query again.

This time the results look different.

Loan Applications Submitted Today

-------------------------------

Loan #101

Loan #102

Loan #103

Loan #104

Total: 4

Notice what changed.

None of the existing rows were modified.

Instead, an entirely new row appeared.

This is called a Phantom Read.

A phantom read occurs when the same query returns a different set of rows because another transaction inserted, updated, or deleted records that match the query’s search criteria.

Think of it like counting the number of people in a room.

You count 20 people.

While you’re writing the number down, someone walks into the room.

You count again.

Now there are 21 people.

Nothing about the original twenty people changed.

The difference is that a new “phantom” appeared between your two observations.

Timeline of a Phantom Read

Transaction A                     Transaction B

BEGIN

SELECT *

WHERE loan_date = TODAY

Returns 3 rows

                               BEGIN

                               INSERT Loan #104

                               COMMIT

SELECT *

WHERE loan_date = TODAY

Returns 4 rows ❌

COMMIT

Unlike a non-repeatable read, where an existing row changes, phantom reads involve the appearance or disappearance of entire rows.

Why Phantom Reads Matter

Imagine you’re calculating today’s total revenue for financial reporting.

Your reporting transaction begins at 5:00 PM and starts aggregating sales.

While it’s still processing, new sales continue being recorded.

Different parts of the report may now be working with different datasets.

The total revenue calculated on page one might not match the detailed transaction list generated on page five because new rows appeared while the report was still executing.

In reporting systems, this can produce confusing and inconsistent results.

Higher isolation levels solve this problem by ensuring the transaction sees a consistent snapshot of the data throughout its lifetime, even if other transactions continue inserting new rows.

Lost Updates

The final concurrency anomaly is perhaps the most dangerous because it silently discards valid work.

Imagine two warehouse employees looking at the same inventory record.

The system currently shows:

Laptop Stock = 10

Employee A sells one laptop.

Employee B also sells one laptop at almost exactly the same time.

Both employees read the current stock before making their update.

Each calculates the new quantity as:

10 - 1 = 9

Employee A saves.

The inventory becomes:

9

A fraction of a second later, Employee B saves.

The inventory is still:

9

One of the updates has effectively disappeared.

The correct inventory should now be 8, but because both transactions started from the same original value, one update overwrote the other.

This is known as a Lost Update.

Unlike the previous anomalies, nothing appears obviously wrong.

No errors occur.

No constraints are violated.

The database happily accepts both updates.

The problem is that one user’s work has unintentionally replaced another’s.

Lost updates are one of the primary reasons databases provide row locking, optimistic concurrency control, and stronger isolation levels.

Without these protections, applications that receive many simultaneous updates—such as inventory systems, banking platforms, or booking applications—can slowly drift away from reality without anyone noticing.

Now that we understand the problems, the next question is obvious: How do databases prevent them? That’s exactly what we’ll cover in Part 2.”

“Transactions are not just a database feature—they’re one of the fundamental building blocks of reliable software.”

Imagine you’re transferring KES 10,000 from your savings account to a friend.

From your perspective, it’s a single action. You tap “Send Money”, authenticate the transaction, and wait for confirmation. Behind the scenes, however, the banking system performs several independent operations. It verifies that you have sufficient funds, deducts the amount from your account, credits your friend’s account, records the transaction in a ledger, updates account balances, and generates a receipt.

Each of these operations is important, but together they represent a single business action: a money transfer.

Now imagine the server crashes immediately after deducting the money from your account but before crediting your friend.

The result is disastrous. Your balance has decreased, your friend never receives the money, and unless additional recovery mechanisms exist, the system is left in an inconsistent state. From a customer’s perspective, the money has simply disappeared.

The same problem appears outside banking.

An e-commerce application might create an order, reduce inventory, process a payment, generate an invoice, and send a confirmation email. If the payment succeeds but the order creation fails, the customer has paid for a product that the system doesn’t believe exists. Likewise, in a loan management system, a repayment may update the outstanding balance, post accounting entries, and generate a receipt. If only some of those updates complete, financial records quickly become unreliable.

These problems aren’t caused by bad algorithms or poor business logic. They’re caused by partial execution when only part of a larger operation succeeds.

This is precisely the problem database transactions were designed to solve.

A transaction ensures that multiple database operations behave as a single unit of work. Either every operation succeeds together, or every operation is rolled back as though nothing ever happened. There is no halfway point where your system is left in an inconsistent state.

For backend developers, understanding transactions is just as important as understanding APIs or databases themselves. They are the foundation upon which reliable financial systems, booking platforms, inventory systems, healthcare applications, and countless other business-critical systems are built.

What Is a Database Transaction?

A database transaction is a collection of one or more database operations that the database treats as a single logical operation.

Instead of thinking about individual SQL statements, think about the business process they represent.

Suppose a customer purchases the last laptop in your online store. That single purchase might require your application to:

• Create an order.
• Deduct one item from inventory.
• Reserve the shipment.
• Record the payment.
• Create an invoice.

Although these are separate SQL statements, they represent one business event. Either all of them should succeed, or none of them should.

That’s exactly what a transaction guarantees.

Without transactions, every statement executes independently. If statement number four fails, the previous three remain committed, leaving your data inconsistent.

With transactions, the database waits until you’re satisfied that every operation has completed successfully. Only then are the changes permanently saved.

Understanding Transactions Through a Simple Example

Let’s return to the banking example.

Alice wants to transfer KES 10,000 to Bob.

A simplified version of the SQL might look like this:

UPDATE accounts
SET balance = balance - 10000
WHERE account_id = 1;

UPDATE accounts
SET balance = balance + 10000
WHERE account_id = 2;

At first glance, this seems perfectly reasonable.

But imagine the database crashes immediately after the first statement executes.

Alice’s balance has already been reduced.

Bob’s balance has not increased.

The system now contains incorrect financial data.

This is why production systems rarely execute related operations independently.

Instead, they wrap them inside a transaction.

BEGIN;

UPDATE accounts
SET balance = balance - 10000
WHERE account_id = 1;

UPDATE accounts
SET balance = balance + 10000
WHERE account_id = 2;

COMMIT;

If every statement succeeds, the database executes the COMMIT, making the changes permanent.

If any statement fails before that point, the application issues a ROLLBACK, and the database restores itself to exactly the state it was in before the transaction began.

To the outside world, it appears as though the failed transfer never happened.

This “all-or-nothing” behavior is what makes transactions so valuable.

The Lifecycle of a Transaction

Every transaction follows a predictable lifecycle.

BEGIN
   │
Execute SQL Operations
   │
Everything Successful?
   │
 ┌─ Yes ─────────────┐
 │                   │
COMMIT          Changes Saved
 │
 └─ No ──────────────┐
                     │
                 ROLLBACK
                     │
          Database Restored

The process begins with BEGIN, which tells the database to temporarily hold all modifications rather than immediately committing them.

The application then performs one or more operations. These could involve inserting records, updating balances, deleting data, or modifying relationships between tables.

If every operation succeeds, the application calls COMMIT. At that point, the database permanently saves all changes.

If anything goes wrong along the way, perhaps a validation error, a database constraint violation, or even an unexpected server failure, the transaction is rolled back, discarding every change made since BEGIN.

The beauty of this model is that the database itself guarantees consistency. Developers don’t need to manually undo every failed operation because the database handles that responsibility.

The Four ACID Properties

When developers discuss transactions, you’ll almost always hear the term ACID.

Despite sounding intimidating, ACID simply describes the guarantees that modern relational databases provide when executing transactions.

Atomicity

Atomicity means that a transaction is indivisible.

Either every operation succeeds, or none of them do.

Returning to our banking example, it makes no sense for money to be deducted from one account without being added to another. The transaction must succeed completely or fail completely.

Think of flipping a light switch.

The light cannot be half on.

Similarly, a transaction cannot be half completed.

Consistency

Consistency ensures that every transaction leaves the database in a valid state.

Business rules should always remain true.

If your application enforces that inventory can never become negative, then a successful transaction should never violate that rule.

Likewise, if your accounting system requires every journal entry to balance, no committed transaction should ever leave the ledger unbalanced.

Consistency isn’t about preventing bugs in your application logic; it’s about ensuring that completed transactions respect the rules defined by your database and your business.

Isolation

Isolation becomes important when multiple users interact with the system simultaneously.

Imagine two customers attempting to purchase the last available ticket for a concert.

Without proper isolation, both requests may read the inventory before either updates it. Both believe the ticket is available, and both complete the purchase.

You’ve now sold the same seat twice.

Isolation ensures that concurrent transactions don’t interfere with one another in ways that produce inconsistent results.

In our previous article, we discussed race conditions situations where multiple requests compete to modify the same data. Isolation is one of the database mechanisms used to prevent those concurrency problems.

We’ll explore isolation levels in greater depth in the next article because they deserve an entire discussion of their own.

Durability

Durability guarantees that once a transaction has been committed, the changes are permanent.

Even if the server loses power immediately after the commit, the database ensures that committed data survives.

Modern databases achieve this through techniques such as write-ahead logging, transaction logs, and crash recovery.

For developers, the important takeaway is simple: once the database confirms a successful commit, you can trust that the data has been safely stored.

Transactions Solve Partial Failures: Not Every Problem

One misconception among newer developers is that transactions magically solve every data consistency problem.

They don’t.

Transactions protect against partial execution.

Suppose your application updates three tables and crashes after updating the second one. A transaction ensures that the database rolls everything back, preventing inconsistent data. However, transactions don’t automatically solve concurrency problems.

Imagine two users attempting to withdraw money from the same account simultaneously. Each transaction independently checks the balance before either completes. If both see the same balance and both proceed, the final result may still be incorrect depending on your isolation level. This isn’t a transaction problem. It’s a concurrency problem.

That’s why understanding race conditions and transactions together is so important. They solve different classes of reliability issues.

Common Places You’ll Use Transactions

Transactions appear almost everywhere in modern backend systems.

Payment processing is perhaps the most obvious example. Charging a customer’s card, recording the payment, updating invoices, and generating accounting entries should either all succeed or all fail together.

Inventory management systems use transactions to ensure stock counts remain accurate even when multiple customers are purchasing products simultaneously.

Booking platforms rely on transactions to reserve hotel rooms, airline seats, or event tickets without creating conflicting reservations.

Loan management systems use transactions when posting repayments, updating outstanding balances, calculating accrued interest, and recording accounting entries.

Healthcare systems use them to ensure patient records, prescriptions, billing information, and appointment schedules remain synchronized.

Any time a business operation spans multiple database changes, a transaction is usually involved.

Common Mistakes Developers Make

One of the most common mistakes is keeping transactions open for too long.

Imagine starting a transaction, calling a third-party payment API, waiting several seconds for a response, and only then committing the transaction.

During that entire period, database resources may remain locked, reducing performance for other users.

A better approach is to perform external API calls before starting the transaction whenever possible, keeping the transaction focused solely on database operations.

Another common mistake is assuming transactions automatically protect against concurrent updates. As we’ve already seen, concurrency introduces an entirely different set of challenges that require locking strategies or appropriate isolation levels.

Finally, developers sometimes forget that transactions should represent business operations not individual SQL statements. Wrapping every single query in its own transaction rarely provides meaningful benefits.

Transactions in Modern Frameworks

Fortunately, most frameworks make transactions straightforward to use.

Laravel offers the DB::transaction() helper.

Django provides transaction.atomic().

Entity Framework supports BeginTransactionAsync().

Spring Boot uses the @Transactional annotation.

Although the syntax differs, the underlying principle never changes. The framework simply tells the database when to begin the transaction, when to commit it, and when to roll it back if something goes wrong.

Understanding the concept matters far more than memorizing framework-specific syntax.

Transactions, Idempotency, and Race Conditions

If you’ve been following this series, you may have noticed that each concept addresses a different reliability challenge.

Idempotency protects against duplicate requests by ensuring that repeating the same request doesn’t produce duplicate side effects.

Race conditions occur when multiple requests compete to modify shared data simultaneously, leading to unpredictable outcomes.

Transactions ensure that a group of related database operations either all succeed together or all fail together.

Reliable backend systems typically rely on all three.

Imagine a payment API.

Idempotency prevents customers from being charged twice if they retry a request.

Transactions ensure that charging the customer, recording the payment, and updating account balances either all succeed or all fail.

Proper concurrency control ensures that two simultaneous payment requests don’t corrupt shared data.

Each concept complements the others rather than replacing them.

Final Thoughts

Transactions are one of the reasons relational databases remain so powerful. They provide developers with a reliable mechanism for preserving data integrity even when failures occur.

As systems become larger and more distributed, failures become inevitable. Servers crash, networks fail, APIs time out, and users submit requests simultaneously. Transactions don’t eliminate those realities, but they ensure your database remains consistent when they happen.

Whenever you’re implementing a feature that modifies multiple pieces of related data, pause for a moment and ask yourself:

What happens if this operation fails halfway through?

If the answer is “my system ends up in an inconsistent state,” then you’ve almost certainly found a place where a database transaction belongs.

In the next article, we’ll build on this foundation by exploring database isolation levels and why two perfectly valid transactions can still interfere with one another when they run at the same time.

The website shows:

Only 1 ticket remaining.

At exactly the same moment, someone else clicks Buy.

Both of you complete payment.

Both of you receive confirmation emails.

But there was only one ticket.

How did two people successfully purchase the same seat?

Now imagine the same thing happening in a banking application.

Two ATM withdrawals happen at almost the same time.

Both check the balance before either transaction finishes.

Both think there’s enough money.

Both approve the withdrawal.

The account ends up with a negative balance.

Neither application is necessarily “broken.”

Instead, they suffer from one of the most common problems in software engineering:

Race conditions.

Race conditions are among the hardest bugs to reproduce because they don’t happen every time. They may only appear under heavy traffic, high concurrency, or perfect timing. Everything works during testing until your application reaches production.

In this article, we’ll explore what race conditions are, why they happen, how they relate to idempotency, and the techniques developers use to prevent them.

What Is a Race Condition?

A race condition occurs when two or more operations access and modify the same piece of data at the same time, and the final result depends on the order in which they execute.

The important phrase is:

The outcome depends on timing.

That’s what makes race conditions so dangerous.

Sometimes everything works.

Sometimes everything breaks.

The exact same code can produce different results simply because two requests happened a few milliseconds apart.

A Simple Analogy

Imagine two people standing in front of a cookie jar.

The jar contains exactly one cookie.

Both people look inside.

Both see one cookie.

Both reach in.

Both believe they’ll get the cookie.

Reality says otherwise.

Only one cookie exists.

The mistake happened because both people checked the state before either updated it.

Software behaves exactly the same way.

A Real Banking Example

Suppose an account has:

Balance = $100

Two withdrawal requests arrive simultaneously.

Request A → Withdraw $80

Request B → Withdraw $50

Without synchronization:

Request A
↓

Read Balance ($100)

------------

Request B

↓

Read Balance ($100)

------------

Request A

Balance = $20

------------

Request B

Balance = $50

Depending on timing, the final balance might be:

$20

or

$50

or

-$30

None of these outcomes are guaranteed.

This is a race condition.

Why It Works During Development

Most developers test applications alone.

One request.

One browser.

One user.

Everything works perfectly.

Production is different.

Imagine:

• 5,000 users
• Hundreds of requests per second
• Multiple application servers
• Database replication
• Network latency

The probability of two requests colliding becomes much higher.

Race conditions often appear only after an application becomes successful.

Ironically, scaling your application can reveal bugs that never existed during development.

Race Conditions vs Idempotency

If you’ve read my previous article on idempotency, if not, read it first here. You might wonder whether they’re the same thing.

They’re related, but they solve different problems.

Idempotency answers:

What if the same request is sent twice?

Example:

POST /payments

↓

Retry

↓

POST /payments

The solution is an Idempotency-Key.

The same request produces the same result.

Race conditions answer:

What if different requests happen at the same time?

Example:

User A buys last ticket

↓

User B buys last ticket

These are two legitimate requests from different users.

An idempotency key won’t help because the requests are not duplicates.

Race conditions require synchronization, not deduplication.

Real-World Examples

Airline Seat Booking

Only one seat remains.

Two customers purchase it simultaneously.

Without proper locking:

• Seat sold twice
• Refund required
• Customer frustration

E-Commerce Inventory

Stock:

Laptop

Quantity = 1

Two customers purchase simultaneously.

Without protection:

Inventory becomes:

-1

Now you’ve sold a product you don’t have.

Loan Approval Systems

Imagine two loan officers reviewing the same application.

Officer A:

Approve.

Officer B:

Reject.

If both updates happen simultaneously without coordination, the final loan status depends entirely on timing.

Coupon Redemption

Promotion:

First 100 customers only

If multiple requests update the redemption count simultaneously, you might accidentally issue:

103

105

110

coupons.

How Race Conditions Happen

Most race conditions follow this pattern:

Read

↓

Modify

↓

Write

The danger lies between Read and Write.

Example:

Read Balance

↓

Calculate New Balance

↓

Update Balance

If another request changes the balance between those steps, your calculation becomes outdated.

This is known as a Lost Update problem.

Solution 1: Database Transactions

Transactions ensure multiple operations succeed or fail together.

Example:

BEGIN;

SELECT balance
FROM accounts
WHERE id = 1;

UPDATE accounts
SET balance = balance - 80
WHERE id = 1;

COMMIT;

Transactions protect data consistency.

However, they don’t automatically eliminate every race condition.

Isolation levels matter too.

Solution 2: Row-Level Locks

Many relational databases allow locking a row while it’s being updated.

Example:

SELECT *
FROM accounts
WHERE id = 1
FOR UPDATE;

Now:

Request A locks the row.

Request B must wait.

Only after Request A finishes can Request B continue.

This guarantees consistent updates.

Solution 3: Optimistic Locking

Instead of preventing conflicts, optimistic locking detects them.

Imagine a version number.

Account

Balance = 100

Version = 5

Update:

WHERE Version = 5

If another request updates the row first:

Version = 6

Your update fails.

The client retries with fresh data.

Optimistic locking works well when collisions are relatively rare.

Solution 4: Distributed Locks

What if your application runs on multiple servers?

Server A

↓

Database

↑

Server B

A normal in-memory lock won’t work because each server has its own memory.

Instead, developers use distributed locking systems like:

• Redis (Redlock)
• ZooKeeper
• etcd
• Consul

These coordinate access across multiple application instances.

Solution 5: Atomic Database Operations

Sometimes you don’t need to:

Read

↓

Calculate

↓

Write

Instead, let the database perform the update atomically.

Bad:

SELECT quantity;

quantity--;

UPDATE products;

Better:

UPDATE products

SET quantity = quantity - 1

WHERE quantity > 0;

Now the database guarantees consistency.

Detecting Race Conditions

One reason race conditions are difficult is that they rarely appear during manual testing.

Ways to uncover them include:

• Load testing with concurrent users
• Stress testing
• Running parallel integration tests
• Simulating delayed responses
• Chaos engineering
• Monitoring production logs

If a bug only appears “sometimes,” concurrency should be one of your first suspects.

Best Practices

When designing systems that handle shared data:

• Keep transactions short.
• Avoid long-running locks.
• Prefer atomic database operations where possible.
• Use optimistic locking when contention is low.
• Use pessimistic locking for critical resources.
• Test with concurrent requests, not just sequential ones.
• Understand your database’s transaction isolation levels.

Most importantly, always ask:

“What happens if two users do this at exactly the same time?”

Final Thoughts

Race conditions aren’t caused by bad developers, they’re caused by systems becoming concurrent.

As applications grow, users interact simultaneously, background jobs overlap, and services communicate in parallel. Timing becomes unpredictable.

That’s why building reliable software isn’t just about writing correct logic for one request. It’s about ensuring your logic remains correct when hundreds or thousands of requests happen together.

If idempotency protects you from duplicate requests, race condition handling protects you from competing requests.

Together, they form two of the most important building blocks for designing resilient APIs and distributed systems.

The next time you write code that reads, modifies, and writes shared data, pause for a moment and ask:

“What happens if someone else does this at the exact same time?”

If you don’t know the answer, you’ve just found your next engineering problem to solve.

You click the “Pay Now” button.

Nothing happens.

After a few seconds, you assume the request failed, so you click the button again.

And again.

A few minutes later, you discover you’ve been charged three times.

What happened?

From the user’s perspective, the payment seemed to fail. From the server’s perspective, however, it successfully processed every request it received.

This is one of the most common problems in distributed systems, and it’s exactly why idempotency exists.

Whether you’re building payment systems, booking platforms, inventory management software, or any API that changes data, retries are inevitable. Networks fail, clients time out, mobile connections drop, and users double-click buttons.

A well-designed API should survive these retries without creating duplicate side effects.

In this article, we’ll explore what idempotency is, why it matters, and how to implement it in your own APIs.

What Is Idempotency?

In simple terms, an idempotent operation can be performed multiple times without changing the final result beyond the first successful execution.

For example:

Turn on the light.
Turn on the light again.
Turn on the light again.

The light is still ON.

Nothing new happened after the first request.

The final state remains the same.

That’s idempotency.

Now compare it with this:

Deposit $100
Deposit $100
Deposit $100

Your account balance increases by $300.

This operation is not idempotent because every request changes the system.

Why Retries Happen

Many developers assume users only send one request.

Reality is different.

Requests are retried because of:

• Slow internet connections
• Gateway timeouts
• Reverse proxies
• Mobile network interruptions
• Browser refreshes
• Double-clicking buttons
• Client retry mechanisms
• Load balancers
• Microservice communication failures

Imagine this timeline:

Client -------- POST /payments --------> API

             Payment succeeds

API -------- 200 OK --------X

(Response never reaches client)

Client waits...

Client retries.

POST /payments again.

The client believes the payment failed.

The server already completed it.

Without idempotency…

The payment happens twice.

HTTP Methods and Idempotency

HTTP itself distinguishes between idempotent and non-idempotent methods.

GET

GET /users/10

Read the user.

Call it once.

Call it 100 times.

Nothing changes.

Idempotent

PUT

PUT /users/10
{
   "name": "Billy"
}

Replacing the same resource repeatedly produces the same result.

Idempotent

DELETE

DELETE /users/10

Delete the user.

Deleting an already deleted user doesn’t delete them twice.

The final state is still:

User does not exist.

Idempotent

POST

POST /orders

Create a new order.

Call it twice.

You now have two orders.

Not idempotent

This is why POST requests often require additional protection.

Why Payment APIs Use Idempotency Keys

Payment providers like Stripe popularized the use of Idempotency Keys.

The idea is simple.

The client generates a unique identifier.

Example:

Idempotency-Key:
6ab89d3b-acde-4d71-b20d-483d8d0ef091

Every retry sends the same key.

POST /payments

Idempotency-Key:
6ab89d3b-acde-4d71-b20d-483d8d0ef091

When the server receives the request:

1. Check if this key already exists.
2. If not, process the payment.
3. Save both the key and the response.
4. Return the response.

If the same request arrives again with the same key:

Instead of charging the customer again…

Return the previously stored response.

Client

POST /payments
Key: ABC123

↓

Server

Charge customer

↓

Store

ABC123 → Payment #456

↓

Return success

---

Retry

POST /payments
Key: ABC123

↓

Lookup

ABC123 exists

↓

Return Payment #456

No second charge.

Implementing Idempotency

A common workflow looks like this.

Step 1

Receive request.

POST /orders

Headers

Idempotency-Key:
XYZ987

Step 2

Search database.

SELECT *
FROM idempotency_keys
WHERE key = 'XYZ987'

Found?

Yes.

Return stored response.

Done.

Step 3

Not found?

Create the resource.

Create Order

Step 4

Store:

Key

Response

Status Code

Timestamp

Now every retry returns the same response.

Example in Node.js (Express)

app.post("/payments", async (req, res) => {
    const key = req.header("Idempotency-Key");

    const existing = await Idempotency.findOne({ key });

    if (existing) {
        return res.status(existing.status).json(existing.response);
    }

    const payment = await processPayment(req.body);

    await Idempotency.create({
        key,
        status: 201,
        response: payment,
    });

    return res.status(201).json(payment);
});

Python (FastAPI)

from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse

app = FastAPI()

@app.post("/payments")
async def create_payment(request: Request):
    key = request.headers.get("Idempotency-Key")

    existing = await Idempotency.find_one(key=key)

    if existing:
        return JSONResponse(
            content=existing.response,
            status_code=existing.status,
        )

    body = await request.json()
    payment = await process_payment(body)

    await Idempotency.create(
        key=key,
        status=201,
        response=payment,
    )

    return JSONResponse(content=payment, status_code=201)

C# (ASP.NET Core)

app.MapPost("/payments", async (
    HttpRequest request,
    IdempotencyStore store,
    PaymentService payments) =>
{
    var key = request.Headers["Idempotency-Key"].ToString();

    var existing = await store.FindAsync(key);
    if (existing is not null)
    {
        return Results.Json(existing.Response, statusCode: existing.Status);
    }

    var body = await request.ReadFromJsonAsync<PaymentRequest>();
    var payment = await payments.ProcessAsync(body!);

    await store.CreateAsync(new IdempotencyRecord(key, 201, payment));

    return Results.Json(payment, statusCode: StatusCodes.Status201Created);
});

Go

func createPayment(w http.ResponseWriter, r *http.Request) {
    key := r.Header.Get("Idempotency-Key")

    existing, err := idempotency.FindOne(r.Context(), key)
    if err == nil && existing != nil {
        w.WriteHeader(existing.Status)
        json.NewEncoder(w).Encode(existing.Response)
        return
    }

    var body PaymentRequest
    if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    payment, err := processPayment(r.Context(), body)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    if err := idempotency.Create(r.Context(), IdempotencyRecord{
        Key:      key,
        Status:   http.StatusCreated,
        Response: payment,
    }); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    w.WriteHeader(http.StatusCreated)
    json.NewEncoder(w).Encode(payment)
}

Laravel

Route::post('/payments', function (Request $request) {
    $key = $request->header('Idempotency-Key');

    $existing = Idempotency::where('key', $key)->first();

    if ($existing) {
        return response()->json($existing->response, $existing->status);
    }

    $payment = processPayment($request->all());

    Idempotency::create([
        'key' => $key,
        'status' => 201,
        'response' => $payment,
    ]);

    return response()->json($payment, 201);
});

The logic is surprisingly simple.

The complexity comes from storing and managing the keys correctly.

Where Should Idempotency Keys Be Stored?

Options include:

Database

Best for most applications.

Pros:

• Persistent
• Reliable
• Easy to query

Cons:

• Slightly slower

Redis

Excellent for high-volume APIs.

Pros:

• Extremely fast
• TTL support
• Easy expiration

Many APIs automatically expire keys after 24 hours.

In-Memory

Useful only during development.

Not recommended for production.

Restarting the server loses everything.

Common Mistakes

Reusing Keys

Every logical operation should have its own unique key.

Bad:

ABC123

used today

used tomorrow

Good:

New checkout

↓

Generate new UUID

Ignoring Request Differences

Suppose the first request is:

$50

The retry is:

$500

Same key.

Different body.

The server should reject this request because the key is being reused for a different operation.

Never Expiring Keys

Keeping millions of old keys forever wastes storage.

Most APIs expire them after:

• 24 hours
• 48 hours
• 7 days

depending on business requirements.

Real-World Use Cases

Idempotency is valuable anywhere duplicate requests could have costly consequences.

Examples include:

• Payment processing
• Bank transfers
• Order creation
• Hotel reservations
• Flight bookings
• Ticket purchases
• Subscription billing
• Inventory updates
• Webhook processing
• Email sending
• Message queues

If performing the same action twice could create an incorrect outcome, idempotency is worth considering.

When You Don’t Need Idempotency

Not every endpoint needs an idempotency key.

For example:

GET /posts

No state changes.

No duplicates.

No problem.

Likewise, endpoints such as:

• Search
• Filtering
• Reading reports
• Viewing profiles

are already naturally idempotent.

Reserve idempotency mechanisms for operations where retries could create unintended side effects.

Final Thoughts

Idempotency isn’t just an implementation detail—it’s a reliability feature.

In distributed systems, retries are normal. Networks are unreliable, users click buttons more than once, and clients retry requests automatically. Instead of hoping those situations never happen, design your APIs to handle them gracefully.

By using idempotency keys, storing responses, validating retries, and choosing the right storage strategy, you can prevent duplicate orders, repeated payments, and other costly errors.

A resilient API isn’t one that never receives duplicate requests.

It’s one that produces the correct outcome even when duplicate requests inevitably arrive.

The next time you design a POST endpoint, ask yourself:

“What happens if this request is sent twice?”

If the answer is “something bad,” it’s probably time to add idempotency.

I wasn’t thinking about traffic, search engine rankings, personal branding, or becoming a better writer. I simply wanted a place to share my thoughts, document what I was learning, and build something that belonged to me.

Like many developers, I started blogging because I was learning new things every day and wanted to keep track of my progress. At the time, I had no idea how much writing would teach me—not just about technology, but about consistency, communication, patience, and personal growth.

Now, after publishing my first 72 blog posts, I’ve learned lessons that extend far beyond blogging itself. Some lessons came from successes, while others came from mistakes and moments of frustration.

In this article, I want to share the most important lessons I’ve learned from publishing my first 72 blog posts.

1. Consistency Beats Perfection

One of the biggest mistakes I made early on was trying to make every article perfect.

I would spend hours tweaking sentences, rewriting paragraphs, and overthinking every detail. Sometimes I delayed publishing because I felt an article wasn’t “good enough.”

Over time, I realized that perfection is the enemy of progress.

The articles that helped me grow weren’t necessarily my best articles. They were simply the articles I published.

Consistency creates momentum.

Publishing one article every week for a year is far more valuable than spending months trying to create a single masterpiece.

The biggest breakthrough came when I stopped chasing perfection and started focusing on showing up consistently.

2. Writing Clarifies Thinking

Many times, I thought I understood a topic until I tried to explain it in writing.

That’s when the gaps in my knowledge became obvious.

Writing forced me to organize my thoughts, simplify complex ideas, and identify areas where my understanding was incomplete.

The process taught me that learning and explaining are not the same thing.

If I couldn’t explain a concept clearly, I probably didn’t understand it as well as I thought.

This lesson made me a better learner and a better developer.

3. Nobody Reads Your First Posts—And That’s Okay

One of the hardest realities for new bloggers is that almost nobody reads your early content.

I remember publishing articles and checking analytics repeatedly, hoping to see visitors.

Most of the time, there were very few.

At first, that felt discouraging.

But eventually I realized something important:

The purpose of your first blog posts isn’t to attract thousands of readers.

The purpose is to learn how to write.

Your first posts are practice.

Every article improves your skills, your confidence, and your ability to communicate.

The audience comes later.

4. The Habit Matters More Than Motivation

Motivation is unreliable.

Some days you’ll feel inspired.

Other days you’ll have no desire to write.

If blogging depends entirely on motivation, consistency becomes impossible.

One of the most valuable lessons I learned is that habits outperform motivation.

The writers who succeed aren’t necessarily the most talented.

They’re the ones who continue writing even when they don’t feel like it.

Creating a writing habit transformed blogging from an occasional activity into a regular part of my routine.

5. Every Post Doesn’t Need to Be Revolutionary

Early in my blogging journey, I believed every article needed a unique insight or groundbreaking idea.

I was wrong.

Many of my most useful posts covered topics that had already been discussed countless times.

The difference was that I shared my own perspective and experience.

Your value doesn’t come from inventing entirely new ideas.

It comes from explaining ideas through your own lens.

There’s always someone who needs the explanation that only you can provide.

6. Writing Builds Confidence

Publishing online can feel intimidating.

You’re sharing your thoughts with strangers.

You’re exposing your ideas to criticism.

You’re putting your work in public.

At first, this can be uncomfortable.

But over time, each article builds confidence.

You become more comfortable expressing your opinions.

You stop worrying about being perfect.

You gain confidence in your ability to communicate and contribute valuable ideas.

That confidence eventually extends beyond blogging into other areas of life and work.

7. Blogging Is a Long-Term Game

One of the most important lessons I learned is that blogging rewards patience.

Results rarely happen overnight.

Traffic grows slowly.

Authority builds gradually.

Opportunities emerge unexpectedly.

Many people quit because they expect immediate results.

The reality is that blogging is similar to investing.

Small contributions made consistently over time eventually compound into significant outcomes.

The key is staying committed long enough to experience those benefits.

8. Publishing Teaches More Than Consuming

Before I started blogging, I spent most of my time consuming content.

I watched tutorials.

I read articles.

I followed courses.

While these activities were helpful, publishing taught me far more.

Creating content forces you to think deeply, research thoroughly, and communicate clearly.

It transforms you from a consumer into a creator.

That shift changed the way I learn forever.

9. Your Blog Becomes a Personal Archive

One unexpected benefit of blogging is having a permanent record of your journey.

Looking back at my older articles shows how much I’ve grown.

I can see:

• How my thinking evolved.
• What I was learning.
• Problems I was solving.
• Mistakes I made.

The blog becomes more than a collection of articles.

It becomes a timeline of personal growth.

10. Small Efforts Compound Over Time

When I published my first article, it felt insignificant.

The same was true for my second, third, and tenth article.

But by the time I reached fifty posts, those small efforts had accumulated into something meaningful.

Fifty articles represent:

• Hundreds of hours of learning.
• Hundreds of hours of writing.
• Dozens of lessons learned.
• A growing body of work.

This reminded me that success is rarely the result of a single big action.

It’s usually the result of many small actions repeated consistently over time.

11. The Real Reward Isn’t Traffic

When people think about blogging success, they often think about page views, followers, and analytics.

Those metrics matter, but they’re not the greatest reward.

The greatest reward is who you become during the process.

Blogging made me:

• A better writer.
• A better learner.
• A better communicator.
• A more disciplined creator.
• A more thoughtful developer.

Those benefits are far more valuable than any traffic number.

Looking Ahead

Reaching 72 blog posts feels like an important milestone, but it also feels like the beginning.

There’s still so much to learn.

So many topics to explore.

So many experiences to share.

The goal isn’t simply to publish more articles.

The goal is to continue learning, improving, and documenting the journey.

If the first 72 posts taught me anything, it’s that growth comes from consistency, curiosity, and the willingness to keep showing up.

Conclusion

Publishing my first 72 blog posts taught me lessons that extend far beyond writing.

It taught me patience when growth was slow.

It taught me discipline when motivation disappeared.

It taught me confidence when self-doubt appeared.

Most importantly, it taught me that meaningful progress is often the result of small actions repeated consistently over time.

If you’re thinking about starting a blog, my advice is simple:

Start now.

Don’t wait until you’re an expert.

Don’t wait until everything is perfect.

Write your first post.

Then write your second.

Then your third.

One day, you’ll look back and realize those small steps created something much bigger than you ever imagined.

I did an article about importance of writing for developers which you can check out.

You’ve been staring at the same bug for 2 hours. You’ve restarted the server. Cleared cache. Added random console.logs. Somehow… it still doesn’t work.

At some point, you stop coding and start guessing.

And that’s the real problem.

The issue isn’t the bug. It’s that nobody actually teaches debugging as a skill.

The Way Most Developers Debug

Let’s be honest. Most of us learned debugging like this:

• Sprinkle console.log everywhere
• Change random lines and hope something works
• Copy-paste error messages into Google
• Restart everything “just in case”

It sometimes works.

But it’s slow, frustrating, and unreliable.

It’s not debugging.

It’s trial and error disguised as progress.

What Debugging Actually Is

Here’s the mindset shift that changes everything:

Debugging is not about fixing code. It’s about finding where your mental model diverges from reality.

You think the system works one way.

Reality says otherwise.

Your job is to close that gap.

The Debugging Mindset

Before tools, before techniques, this is what matters most.

1. Assume Your Assumptions Are Wrong

If something doesn’t work, at least one thing you believe is false.

Your job is to find it.

2. Narrow the Problem Space

Bad debugging:

“Something is wrong with the app”

Good debugging:

“The issue happens only when this function runs after login”

3. Reproduce Before Fixing

If you can’t reliably reproduce the bug, you don’t understand it.

And if you don’t understand it, your fix is luck—not skill.

4. One Change at a Time

If you change 5 things and it works… which one fixed it?

You don’t know.

That’s how bugs come back later.

5. Understand Before You Patch

Quick fixes feel good.

Understanding the root cause makes you dangerous (in a good way).

A Repeatable Debugging Process

This is where things become practical.

Step 1: Reproduce the Bug

Make it happen consistently.

Click button → error appears  
Refresh → still happens  
Different browser → still happens

If it’s inconsistent, your first task is to find the pattern.

Step 2: Define Expected vs Actual

Write it down clearly.

Expected: API returns user data  
Actual: API returns empty array

This step alone eliminates confusion.

Step 3: Isolate the Problem

Shrink the scope.

• Comment out unrelated code
• Remove layers (UI → API → DB)
• Test pieces independently

Think of it like this:

[ UI ] → [ API ] → [ Database ]

Which layer is lying?

Step 4: Form a Hypothesis

Be explicit:

“I think the API is returning empty data because the query filter is wrong.”

Now you’re not guessing—you’re testing a theory.

Step 5: Test the Hypothesis

Use targeted tools:

• Logs
• Breakpoints
• Network inspector

Example:

console.log("User ID:", userId)

But intentional—not random.

Step 6: Fix and Verify

Fix it.

Then confirm:

• Does it work in all cases?
• Did you break something else?

Step 7: Understand the Root Cause

This is where most devs stop too early.

Don’t just fix it—explain it:

“The bug happened because the state updated asynchronously, and we read it too early.”

Now you’ve learned something reusable.

Real Example: “The API Is Broken” (But It’s Not)

Let’s walk through a real scenario.

The Bug

Frontend shows: No data available

Initial Assumption

“The API is broken.”

Step 1: Check Network Tab

You open DevTools → Network:

API returns correct data

So… not the API.

Step 2: Check State

console.log(data)

It logs:

[]

Empty array.

Step 3: Trace the Flow

useEffect(() => {
  fetchData()
}, [])

Inside fetchData:

setData(response.data)
console.log(data) // still empty

The Problem

React state updates are asynchronous.

You’re logging before state updates.

The Fix

useEffect(() => {
  fetchData()
}, [])

useEffect(() => {
  console.log(data)
}, [data])

The Lesson

The bug wasn’t in the API. It was in your mental model of how state updates work.

Tools That Actually Help

Not everything is about tools—but the right ones matter.

1. Browser DevTools (Underrated Powerhouse)

• Network tab → verify API calls
• Console → inspect runtime values
• Application tab → check storage

2. Breakpoints (Game Changer)

Instead of spamming logs:

Pause execution and inspect state live

3. Intentional Logging

Bad:

console.log("here")

Good:

console.log("User after login:", user)

4. Stack Traces

Read them.

They literally tell you:

• Where the error happened
• What triggered it

5. Rubber Duck Debugging

Explain the bug out loud.

Yes, seriously.

You’ll often solve it mid-explanation.

Problem → Hypothesis → Test → Learn → Repeat

Common Debugging Traps

Avoid these and you’ll already be ahead of most devs:

Fixing Symptoms Instead of Causes

You silence the error… but the bug is still there.

Changing Too Many Things at Once

Now you don’t know what worked.

Ignoring Error Messages

The error is literally telling you what’s wrong.

Read it.

Assuming the Bug Is “Weird”

It’s almost never weird.

It’s misunderstood.

“It Works on My Machine”

This is not a flex.

It’s a clue.

A Better Mental Model

Expectation ≠ Reality  
        ↓  
Investigate the gap

That’s debugging.

Final Thought

The best developers aren’t the ones who write perfect code. They’re the ones who can quickly understand why things break.

Debugging isn’t a side skill.

It is the job.

Heads-up: v2 uses different endpoints and JSON than classic Accounts CRUD. Always confirm API version, preview flags, and SDK support in Stripe’s documentation before shipping production code.

What is Stripe Connect?

Stripe Connect is for platforms that move money between buyers, sellers, and your platform. You connect connected accounts to your platform account so you can:

• Collect payments from customers
• Pay out to sellers or service providers
• Take application fees where appropriate

Stripe holds the payments rail; you own product and UX choices.

Two ideas at once: “style” (Standard / Express / Custom) and “shape” (v2 configurations)

The original guide compares Standard, Express, and Custom as who owns onboarding, dashboards, and compliance.

Accounts v2 adds a separate axis: what capabilities a single Account has, expressed as configurations:

You can assign one or more configurations to the same Account. That is the core promise of v2: one identity, multiple roles, less manual ID mapping.

The Standard / Express / Custom choice is still real: it describes OAuth vs Stripe-hosted onboarding vs fully custom UI. On v2 you implement those experiences while creating and updating Accounts through the v2 surface (where supported).

Standard vs Express vs Custom (unchanged product trade-offs)

On v2, you still make these product choices—but you attach merchant / customer / recipient configurations to match what each connected business must do.

Architectural overview

Fund flow is unchanged at a high level:

What changes in v2 is object modeling:

• One Account can represent both “seller” and “buyer of your SaaS” if you add merchant and customer configurations.
• APIs that took customer may accept customer_account with an Account ID—see using Accounts as customers.

Accounts v2 primitives (before code)

Stripe’s v2 Account objects differ from v1’s flat Account create. You typically send:

• identity — country, entity type, business details.
• configuration — nested blocks such as merchant, customer, recipient, each with capabilities you request (card_payments, balance / payout capabilities as named in current docs).
• defaults — currency, locales, responsibilities (fees / losses collector), etc.
• include — ask the API to return nested sections (e.g. configuration.merchant, requirements).

Responses may omit fields unless you include them—see Stripe’s note on includable response values.

API version: v2 calls often require a specific Stripe-Version header (including preview versions while the API evolves). Set this from the exact value in Stripe’s Accounts v2 docs for your integration window.

Implementation: creating an Account on v2 (HTTP-first)

Official examples are REST + JSON. Below is illustrative—copy field names, capability keys, and version from the current docs, not from this blog alone.

cURL (canonical pattern from Stripe)

Stripe documents POST /v2/core/accounts with a JSON body. Conceptually:

curl -X POST https://api.stripe.com/v2/core/accounts \
  -H "Authorization: Bearer sk_test_..." \
  -H "Stripe-Version: <version-from-stripe-docs>" \
  -H "Content-Type: application/json" \
  --data '{
    "contact_email": "seller@example.com",
    "display_name": "Example Seller",
    "identity": {
      "country": "us",
      "entity_type": "company",
      "business_details": { "registered_name": "Example Co" }
    },
    "configuration": {
      "merchant": {
        "capabilities": {
          "card_payments": { "requested": true }
        }
      },
      "customer": {
        "capabilities": {
          "automatic_indirect_tax": { "requested": true }
        }
      }
    },
    "defaults": {
      "currency": "usd",
      "responsibilities": {
        "fees_collector": "stripe",
        "losses_collector": "stripe"
      },
      "locales": ["en-US"]
    },
    "include": [
      "configuration.customer",
      "configuration.merchant",
      "identity",
      "requirements"
    ]
  }'

This shows the mental model: one create, multiple configurations, explicit includes.

Python (generic HTTP client)

Until your stripe SDK exposes stable helpers for every v2 path, httpx or requests keeps you aligned with the docs:

import os
import requests

def create_account_v2():
    r = requests.post(
        "https://api.stripe.com/v2/core/accounts",
        headers={
            "Authorization": f"Bearer {os.environ['STRIPE_SECRET_KEY']}",
            "Stripe-Version": "<version-from-stripe-docs>",
            "Content-Type": "application/json",
        },
        json={
            # ...same structure as the cURL example...
        },
        timeout=30,
    )
    r.raise_for_status()
    return r.json()

PHP (Laravel-style)

Use Guzzle or Laravel Http::withHeaders([...])->post(...) with the same URL, Stripe-Version, and JSON body. Keep secrets in config, not in source control.

C# (.NET)

Use HttpClient with StringContent(json, Encoding.UTF8, "application/json") and the same headers. Deserialize the JSON response into your own DTOs that track id, requirements, and configuration.* state.

OAuth / Standard accounts: Stripe currently directs platforms that authenticate with OAuth to connected accounts to continue using v1 for that path. Treat Standard as documented in the v1 guide until your OAuth + v2 story is explicitly supported for your use case—see Accounts v2 and OAuth.

Onboarding links (Express-style flows)

For Express-like experiences you still send users through Stripe-hosted onboarding where the product allows it. With a connected account ID returned from v2 (acct_...), Account Links (v1 resource) remain the usual tool for account_onboarding—the same pattern as the v1 article, but the account value may come from a v2 create. Verify compatibility for your API version in Stripe’s docs.

Python (v1 Account Links API, account id from v2 create):

import stripe
stripe.api_key = os.environ["STRIPE_SECRET_KEY"]

link = stripe.AccountLink.create(
    account=connected_account_id,
    refresh_url="https://yourapp.com/reauth",
    return_url="https://yourapp.com/complete",
    type="account_onboarding",
)

Custom-style integrations on v2

Custom still means: you own KYC collection, ToS acceptance, and ongoing verification. On v2 you express that by:

• Sending complete identity data the API requires.
• Requesting merchant / recipient capabilities your product needs.
• Handling requirements and webhooks the same way you would for Custom on v1—only the payload shape differs.

You may still use tos_acceptance-style fields where the v2 schema maps them; follow Stripe’s v2 reference for exact property names.

Using Accounts as customers (customer_account)

Where you used customer=cus_..., many flows accept customer_account=acct_... for an Account that has the customer configuration. Example pattern from Stripe (conceptual):

curl https://api.stripe.com/v1/setup_intents \
  -u "sk_test_...:" \
  -H "Stripe-Version: <version-from-stripe-docs>" \
  -d customer_account=acct_123 \
  -d "payment_method_types[]=card" \
  -d confirm=true \
  -d usage=off_session

Details and supported objects live under using Accounts as customers.

Checking balances

For many Connect operations, connected account scoping is unchanged. Python:

balance = stripe.Balance.retrieve(stripe_account=account_id)

PHP:

$balance = \Stripe\Balance::retrieve([], ['stripe_account' => $accountId]);

C#:

var balance = await stripe.Balance.GetAsync(
    new BalanceGetOptions(),
    new RequestOptions { StripeAccount = accountId }
);

Confirm in Stripe’s docs whether your v2 account IDs behave identically for every Balance and v1 helper you rely on during migration.

Compliance responsibilities

The Standard / Express / Custom compliance split from the original article still applies who collects KYC and who owns disputes. v2 can reduce duplicate identity collection when you add configurations to an existing Account instead of opening a second Customer record.

Choosing a path

Strategic considerations

• Time-to-market: Standard (when OAuth fits) < Express < Custom.
• API surface: v2 adds configuration discipline—plan for migration from v1, not an eternal split (Stripe discourages maintaining both versions simultaneously). - Reference
• SDKs: Expect to use HTTP for some v2 paths until your language SDK is fully aligned.

Final thoughts

Accounts v2 does not erase Standard / Express / Custom—it repackages how you represent connected users in the API. Start from Connect and the Accounts v2 API, add using Accounts as customers when the same legal entity both sells and buys from your platform, and keep the v1 Connect guide handy for OAuth flows and legacy snippets until you have fully moved.

Either way, you still trade off control, compliance, and complexity—only the object model got a long-overdue upgrade.

“It worked locally. Tests passed. But production still broke.”

If you’re building distributed systems with multiple services and frontends, you’ve likely encountered this (whether using .NET + Angular, Node.js + React, Python + Vue, or any combination):

• A backend change gets deployed
• The frontend suddenly breaks
• No tests warned you

The issue isn’t always logic.

It’s often a broken contract between systems.

The Problem: Silent API Breakages

In a typical setup:

• Service Provider: An API or microservice
• Service Consumer: A frontend, app, or another service
• Communication: JSON over HTTP (or any protocol)

Everything depends on one thing:

The consumer and provider agreeing on what data looks like

A Real Example

Let’s use a .NET API and Angular frontend as an example (though this applies to any tech stack).

API Provider (Initial Version)

public class UserDto
{
    public int Id { get; set; }
    public string Name { get; set; }
    public string Email { get; set; }
}

[HttpGet("{id}")]
public IActionResult GetUser(int id)
{
    return Ok(new UserDto
    {
        Id = 1,
        Name = "Billy",
        Email = "billy@example.com"
    });
}

Consumer Interface (Angular Example)

export interface User {
  id: number;
  name: string;
  email: string;
}

Everything works perfectly.

Then a “Small” Change Happens

public class UserDto
{
    public int Id { get; set; }
    public string FullName { get; set; } // renamed
    public string Email { get; set; }
}

Production Result

user.name //  undefined

The UI breaks.

Why Didn’t Traditional Tests Catch This?

• Unit tests → passed (backend logic is fine)
• Integration tests → passed (they used the old model)
• The API still returns valid JSON

But the contract between systems changed—and traditional tests don’t verify that.

What is Contract Testing?

In Contract Testing, a contract is a formal specification of expected behavior and communication rules between two or more components, services or systems.

Contract testing ensures that your service provider always matches what the consumer expects.

A contract defines:

• Request format
• Response structure
• Required fields
• Data types

In Simple Terms

The consumer (frontend, app, or service) defines expectations The provider (API or service) must satisfy them

Consumer vs Provider

Consumer

• Calls the service/API
• Defines expected structure
• Examples: Angular frontend, React app, mobile app, another microservice

Provider

• Returns the data
• Must not break expectations
• Examples: .NET API, Node.js backend, Python service, GraphQL endpoint

How Contract Testing Works

Instead of relying only on integration tests:

1. Angular defines expectations
2. A contract file is generated
3. .NET verifies the contract

Flow

Consumer Test (e.g., Angular)
      ↓
Generates Contract
      ↓
Saved as JSON/YAML
      ↓
Provider verifies against contract (e.g., .NET API)

Practical Example

(.NET backend + Angular frontend as an example)

Step 1: Define Expectations in Consumer (Angular example)

const expectedUser = {
  id: 1,
  name: "Billy",
  email: "billy@example.com"
};

it("should fetch user correctly", async () => {
  const user = await userService.getUser(1);
  expect(user).toEqual(expectedUser);
});

Generated Contract (Simplified)

{
  "request": {
    "method": "GET",
    "path": "/api/users/1"
  },
  "response": {
    "body": {
      "id": 1,
      "name": "Billy",
      "email": "billy@example.com"
    }
  }
}

Step 2: Verify in Provider (.NET API example)

Install Pact:

dotnet add package PactNet

Provider Test

[Fact]
public void VerifyPact()
{
    var pactVerifier = new PactVerifier();

    pactVerifier
        .ServiceProvider("UserApi", "http://localhost:5000")
        .WithFileSource(new FileInfo("pacts/userapi-angular.json"))
        .Verify();
}

If Provider Breaks the Contract

public string FullName { get; set; }

• The test fails immediately

• You catch the issue before deployment

Where Contract Testing Fits

E2E Tests
(user journeys)

Integration Tests
(service interactions)

Contract Tests 
(API agreements)

Unit Tests
(business logic)

Why This Matters in Real Projects

Safer Refactoring

Change DTOs, schema, or API responses without fear. Contract tests verify nothing broke.

Independent Development

Consumer and provider teams move faster. Changes are caught instantly, not in production.

Faster Debugging

Failures clearly show what broke

Stronger CI/CD Pipelines

Consumer Build → Generate Contract
Provider Build → Verify Contract
Deploy → Only if both pass

This works with any tech stack.

Common Mistakes

Over-Specifying Data

Bad:

"name": "Billy Okeyo"

Good:

"name": "string"

Testing Everything

Only validate fields your frontend actually uses

Ignoring Versioning

Breaking contracts without versioning leads to production issues

Best Practices

Keep Contracts Minimal

Focus only on required fields

Version Your API

/api/v1/users
/api/v2/users

Automate in CI/CD

Contracts should be generated and verified automatically

Use Realistic Data

Avoid unrealistic mocks

Final Takeaway

Unit tests verify logic Integration tests verify systems E2E tests verify user flows

Contract tests verify agreements

“Most production bugs aren’t failures… they’re misunderstandings between systems.”

Contract testing eliminates those misunderstandings.

Examples include:

• Sending emails
• Generating reports
• Processing uploaded files
• Running background analytics
• Sending notifications

Running these tasks during an HTTP request can make your application slow and unreliable.

This is where Celery and Redis come in.

Together they allow you to run background jobs asynchronously without blocking your main application.

The Problem with Synchronous Tasks

Imagine a user submits a request that triggers an operation that takes 10 seconds.

For example:

• Generating a financial report
• Parsing a large document
• Sending multiple emails

If your application processes this synchronously:

User Request → Django → Long Task → Response

The user waits for the entire process to finish.

This leads to:

• Slow responses
• Poor user experience
• Possible request timeouts

Introducing Celery

Celery is a distributed task queue that allows you to run background jobs outside the request-response cycle.

Instead of executing tasks immediately, Django sends the job to a queue.

A worker then processes it asynchronously.

The flow becomes:

User Request
    ↓
Django
    ↓
Queue Task
    ↓
Immediate Response
    ↓
Celery Worker
    ↓
Executes Job

This makes your application fast and scalable.

Why Redis?

Celery requires a message broker to manage task queues.

Redis is commonly used because it is:

• Extremely fast
• Lightweight
• Easy to deploy
• Perfect for queues and caching

Redis stores the tasks until workers pick them up.

Example: Sending Email in the Background

Instead of sending email directly in a Django view:

send_mail(
    "Welcome",
    "Thanks for signing up",
    "noreply@example.com",
    [user.email],
)

You create a Celery task:

from celery import shared_task
from django.core.mail import send_mail

@shared_task
def send_welcome_email(email):
    send_mail(
        "Welcome",
        "Thanks for signing up",
        "noreply@example.com",
        [email],
    )

Then call it asynchronously:

send_welcome_email.delay(user.email)

The user gets an immediate response while the email is processed in the background.

Common Use Cases for Celery

Celery is useful for many production tasks:

Email sending

• Welcome emails
• Notifications
• Password resets

Data processing

• Financial calculations
• AI processing
• Data pipelines

Scheduled tasks

• Daily reports
• Cleaning expired sessions
• Updating analytics

Running Celery in Production

A typical Django production stack might look like this:

Users
  ↓
Nginx
  ↓
Gunicorn
  ↓
Django App
  ↓
Redis (Broker)
  ↓
Celery Workers

Each component plays a role:

• Nginx handles web traffic
• Gunicorn runs Django
• Redis manages task queues
• Celery workers execute background jobs

Scaling Celery Workers

One of Celery’s biggest advantages is scalability.

If tasks increase, you simply add more workers.

celery -A project worker --loglevel=info --concurrency=4

More workers mean faster task processing.

Final Thoughts

Celery and Redis are essential tools for running Django applications at scale.

They allow you to:

• Improve response times
• Handle heavy workloads
• Build scalable architectures
• Run background processing reliably

If your Django application handles tasks that take more than a few seconds, moving them to Celery is one of the best architectural decisions you can make.

Images need previewing. CSV files need parsing. Excel files need validation. Large files need handling. And suddenly your “simple upload” becomes a full feature.

In this guide, we’ll build a practical and production-ready file upload system using:

• Angular (Frontend)
• Django + Django REST Framework (Backend)

We’ll cover:

1. Image uploads with preview
2. CSV uploads and parsing
3. Excel uploads and processing
4. Validation and security best practices

Backend Setup (Django + DRF)

1. Install Dependencies

pip install djangorestframework pillow openpyxl pandas

• pillow → Image processing
• openpyxl → Excel support
• pandas → CSV & Excel parsing

2. Configure Media Files

settings.py

MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')

urls.py (project level)

from django.conf import settings
from django.conf.urls.static import static

urlpatterns = [
    # your urls
]

if settings.DEBUG:
    urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

Part 1: Image Upload

Django Model

from django.db import models

class Profile(models.Model):
    name = models.CharField(max_length=100)
    image = models.ImageField(upload_to='profiles/')

Serializer

from rest_framework import serializers
from .models import Profile

class ProfileSerializer(serializers.ModelSerializer):
    class Meta:
        model = Profile
        fields = '__all__'

View

from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework.parsers import MultiPartParser, FormParser
from rest_framework import status

class ProfileUploadView(APIView):
    parser_classes = [MultiPartParser, FormParser]

    def post(self, request):
        serializer = ProfileSerializer(data=request.data)
        if serializer.is_valid():
            serializer.save()
            return Response(serializer.data, status=status.HTTP_201_CREATED)
        return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

Angular Frontend (Image Upload)

HTML

<input type="file" (change)="onFileSelected($event)" accept="image/*" />
<img *ngIf="previewUrl" [src]="previewUrl" width="200" />
<button (click)="upload()">Upload</button>

Component

selectedFile!: File;
previewUrl: string | ArrayBuffer | null = null;

onFileSelected(event: any) {
  this.selectedFile = event.target.files[0];

  const reader = new FileReader();
  reader.onload = () => {
    this.previewUrl = reader.result;
  };
  reader.readAsDataURL(this.selectedFile);
}

upload() {
  const formData = new FormData();
  formData.append('name', 'Billy');
  formData.append('image', this.selectedFile);

  this.http.post('http://localhost:8000/api/upload/', formData)
    .subscribe(res => console.log(res));
}

Part 2: CSV Upload and Parsing

Django View for CSV

import pandas as pd
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework.parsers import MultiPartParser

class CSVUploadView(APIView):
    parser_classes = [MultiPartParser]

    def post(self, request):
        file = request.FILES['file']

        if not file.name.endswith('.csv'):
            return Response({'error': 'Invalid file type'}, status=400)

        df = pd.read_csv(file)

        # Example processing
        total_rows = len(df)
        columns = list(df.columns)

        return Response({
            'rows': total_rows,
            'columns': columns
        })

Angular CSV Upload

uploadCSV(file: File) {
  const formData = new FormData();
  formData.append('file', file);

  this.http.post('http://localhost:8000/api/upload-csv/', formData)
    .subscribe(res => console.log(res));
}

Part 3: Excel Upload (.xlsx)

Django View for Excel

class ExcelUploadView(APIView):
    parser_classes = [MultiPartParser]

    def post(self, request):
        file = request.FILES['file']

        if not file.name.endswith('.xlsx'):
            return Response({'error': 'Invalid file type'}, status=400)

        df = pd.read_excel(file)

        summary = {
            'rows': len(df),
            'columns': list(df.columns)
        }

        return Response(summary)

Validation & Security Best Practices

1. Limit File Size

In settings.py:

DATA_UPLOAD_MAX_MEMORY_SIZE = 5242880  # 5MB

2. Validate File Type Properly

Don’t rely only on extension.

if file.content_type not in ['image/jpeg', 'image/png']:
    return Response({'error': 'Invalid image format'}, status=400)

3. Rename Uploaded Files

import uuid

def upload_to(instance, filename):
    ext = filename.split('.')[-1]
    return f"uploads/{uuid.uuid4()}.{ext}"

4. Handle Large Files with Streaming

For very large CSV files, process in chunks:

for chunk in pd.read_csv(file, chunksize=1000):
    # process chunk
    pass

Bonus: Returning Processed Data

Sometimes you don’t just upload — you process and return a result file.

Example: Generate processed CSV

from django.http import HttpResponse

response = HttpResponse(content_type='text/csv')
response['Content-Disposition'] = 'attachment; filename="processed.csv"'
df.to_csv(response, index=False)
return response

Final Thoughts

File uploads are not just about saving files.

They are about:

• Validation
• Security
• User experience
• Scalability

When done correctly, they become powerful data pipelines inside your application.

If you’re building admin systems, reporting tools, learning platforms, or fintech dashboards — mastering uploads is essential.

One pattern stood out immediately.

I bought a MacBook, and at the time, it felt sufficient. It solved the problem I had: performance, portability, and a solid development environment. From a purely functional standpoint, nothing else was required.

But that sense of “enough” didn’t persist.

Shortly after, I justified getting an iPhone partly for convenience, partly for the ecosystem. Once that was in place, the Apple Watch began to feel like a logical extension: notifications, health tracking, tighter integration. And somehow, even after all that, an Apple TV entered the setup as if the system was still incomplete.

Individually, each purchase was defensible. Collectively, they formed a pattern I hadn’t consciously designed.

What struck me during this reflection was not the spending itself, but the realization that my definition of completeness kept shifting. Satisfaction behaved like a moving target, always one accessory away.

Later, I came across a concept that explained this behavior with surprising precision: The Diderot Effect.

What Is the Diderot Effect?

The Diderot Effect describes a behavioral phenomenon where acquiring a new item triggers a cascade of related purchases, driven by the need for consistency rather than necessity.

The concept originates from Denis Diderot, an 18th-century philosopher who, after receiving a luxurious new robe, felt compelled to replace much of his existing furniture because it no longer “matched” the new standard he had introduced.

In modern terms, the Diderot Effect is essentially scope creep applied to consumption.

Why the Diderot Effect Resonates Strongly in Tech

Technology ecosystems are uniquely optimized to amplify this effect.

1. Ecosystem Lock-In as a Design Strategy

Modern tech products are rarely designed as standalone components. They are built as interdependent systems:

• Phone ↔ laptop ↔ watch ↔ TV
• Cloud services ↔ subscriptions ↔ hardware
• Accessories that unlock “full functionality”

Once you introduce a single high-tier component into your stack, the rest of your setup begins to feel like technical debt.

2. Identity-Driven Tooling

In tech, tools are not just utilities they are signals. Owning certain devices, editors, keyboards, or setups subtly communicates:

• Competence
• Professionalism
• “Seriousness” about the craft

The internal logic becomes:

If I’m this kind of developer, my tools should reflect that.

This is where the Diderot Effect stops being about objects and starts being about identity coherence.

3. Optimization Culture

Engineers are trained to optimize systems. Unfortunately, that mindset often spills into consumption:

• Latency → upgrade
• Minor friction → replace
• Marginal improvement → justify cost

Not every inefficiency needs a hardware solution but the instinct is deeply ingrained.

Real-World Tech Examples of the Diderot Effect

• Buying a new laptop → external monitor → mechanical keyboard → standing desk
• Upgrading a phone → earbuds → watch → chargers in every room
• Improving a dev setup → paid tools → subscriptions → cloud services

Each step feels incremental. The aggregate cost rarely does.

The Hidden Costs Engineers Often Overlook

1. Financial Leakage Through “Reasonable” Decisions

Most Diderot-driven purchases pass basic rational checks. The problem isn’t irrationality it’s unbounded justification.

2. Perpetual Dissatisfaction

When standards continuously shift upward, stability disappears. You’re always upgrading the environment instead of leveraging it.

3. Loss of Intentional System Design

Your setup evolves reactively rather than architecturally.

Managing the Diderot Effect as a Technical Thinker

The solution isn’t abstinence it’s intentional system design.

1. Introduce Decision Latency

Treat secondary purchases like production changes:

• Add a delay (48 hours, a week, or a sprint)
• Re-evaluate the actual requirement

Most impulses decay with time.

2. Define System Boundaries

Before upgrading, explicitly define the scope:

“This purchase solves this problem. Nothing else changes.”

Boundaries prevent cascade failures.

3. Separate Functional Debt from Aesthetic Debt

Ask:

• Is this blocking productivity?
• Or does it simply look inferior relative to a new baseline?

Most Diderot chains are triggered by aesthetics masquerading as inefficiency.

4. Budget Upgrades as Projects, Not Reactions

Instead of ad-hoc improvements:

• Plan upgrades quarterly or annually
• Allocate fixed budgets
• Close the project when scope is met

No silent expansions.

5. Be Wary of Identity-Based Justifications

Phrases like:

• “At my level, I should have…”
• “This setup doesn’t reflect where I am now…”

These are signals that the Diderot Effect is driving the decision, not necessity.

When the Diderot Effect Can Be Used Intentionally

Not all cascades are bad.

Applied deliberately, the effect can reinforce positive systems:

• Fitness → nutrition → sleep optimization
• Learning → better tools → deeper focus
• Workspace upgrades that genuinely reduce friction

The difference lies in conscious orchestration versus unconscious drift.

Final Thoughts

The Diderot Effect is not a failure of discipline, it’s a predictable system behavior triggered by introducing a higher standard into an existing environment.

In tech, where optimization and cohesion are prized, this effect is especially potent.

Reflecting on my own experience, from a single MacBook purchase to a fully integrated ecosystem made one thing clear: “Enough” must be explicitly defined, or it will keep moving.

As engineers, we design systems for a living. Our consumption habits deserve the same level of intentionality.

If you told me at the beginning of 2025 that I’d spend the year writing about scalable systems, testing strategies, outages, performance optimizations, and developer excuses I would’ve believed you. If you also told me I’d still be debugging things that “worked yesterday,” I would’ve believed you even faster.

This article is a recap of everything I wrote this year: the lessons, the wins, the bugs, and the “how did this even pass code review?” moments. Think of it as a Spotify Wrapped, but for technical articles, minus the judgment (okay, maybe a little).

System Design & Scalability — Because Your App Will Grow (Whether You’re Ready or Not)

Summary: This year, I spent a lot of time talking about scalable backend systems — not because every app needs to handle millions of users, but because every app eventually meets its first “why is the server on fire?” moment.

• How to Build Scalable Backend Systems with Python, C#, PHP and Dart A practical guide to designing systems that won’t collapse the moment your app gets featured on Twitter… or worse, WhatsApp groups.

Key lesson: Scalability isn’t about overengineering — it’s about not regretting your life choices later.

Testing — Trust Issues, But Make Them Automated

Summary: Testing dominated a good part of the year, mainly because nothing builds trust issues faster than code that looks correct but isn’t.

This series walked through testing from the basics to full CI/CD integration — because “it works on my machine” is not a deployment strategy.

• Unit, Integration, and End-to-End Tests — Part 1 Where we learn that tests are not the enemy — flaky tests are.
• Unit Testing in Depth — Part 2 Small tests, big confidence.
• Integration Testing in Depth — Part 3 When components finally talk to each other… and start arguing.
• End-to-End Testing in Depth — Part 4 Testing your app the same way users break it.
• The Testing Pyramid & CI/CD — Part 5 The moment you realize automation saves both time and sanity.

Key lesson: Tests don’t slow you down — debugging production issues does.

Performance & Tooling — Making Code Faster Without Sacrificing Sleep

Summary: Performance optimization showed up in different forms this year — from modern runtimes to build optimizations. Because sometimes the app isn’t slow… it’s just doing unnecessary work very enthusiastically.

• Diving into WebAssembly: What It Is and Why It Matters For when JavaScript alone just isn’t fast enough.
• Tree Shaking in TypeScript Removing code you forgot existed but was still shipped to production.

Key lesson: If users say your app is slow, they’re probably right. The logs just haven’t confessed yet.

Engineering Culture — Because Code Is Written by Humans (Flawed Ones)

Summary: Not everything this year was serious architecture talk. Some articles leaned into the very human side of software development — where excuses are plentiful and accountability is… negotiable.

• Top 10 Developer Excuses When Code Breaks A humorous breakdown of things we’ve all said — and what actually went wrong.

Key lesson: If you’ve never blamed the cache, the network, or “some weird edge case,” are you even a developer?

Industry Case Studies — Learning the Hard Way (So You Don’t Have To)

Summary: One of the most impactful pieces this year was breaking down a real-world outage — because nothing teaches engineering humility like watching large systems fail in creative ways.

• What Engineers Can Learn From the Cloudflare Outage A reminder that one bad configuration can humble the biggest companies.

Key lesson: Distributed systems don’t fail loudly — they fail politely, globally, and at the worst possible time.

What 2025 Really Taught Me

If I had to summarize the year in engineering truths:

• Scalability matters before users complain.
• Tests are cheaper than apologies.
• Performance issues hide in plain sight.
• Systems fail — preparation determines whether you panic or recover.
• Developers are predictable creatures with unpredictable bugs.

Looking Ahead

In the next year, expect:

• More real-world case studies
• Deeper dives into distributed systems and cloud patterns
• Practical articles you can actually apply on Monday morning

Thank you for reading, sharing, and occasionally finding bugs in my examples (yes, I see you).

Here’s to another year of writing code, fixing mistakes, and pretending we knew what we were doing all along.

On November 18, 2025, the internet shook a little.

Cloudflare, the massive networking and security platform powering millions of websites globally, experienced a major outage that resulted in widespread 5xx errors across the internet. For several hours, key services like CDN routing, Workers KV, Access authentication, and even Cloudflare’s own dashboard were degraded.

In their official incident report, Cloudflare broke down exactly what happened, and the root cause was surprisingly small and deceptively simple:

A configuration file grew larger than expected, violated a hidden assumption in the proxy code, and triggered runtime panics across the global edge.

This incident is a powerful case study in modern distributed systems. Let’s break down what went wrong — and why engineers everywhere should take note.

What Actually Happened? A Chain Reaction From One Oversized File

The root cause began with a permissions change in Cloudflare’s ClickHouse database cluster, which led to duplicate rows in a dataset used by their Bot Management engine. That duplication caused the generated “feature file”, a config-like file that proxies rely on to double in size.

Here’s where assumptions came back to haunt them:

• Cloudflare’s proxy engine expected a maximum of 200 features.
• The new file exceeded that limit.
• An .unwrap() in their Rust-based FL2 proxy assumed the file would always be valid.
• That assumption failed — the code panicked — resulting in cascading 5xx failures.

As Cloudflare noted in their report, this caused two types of breakage across their edge:

1. FL2 proxies (new engine): Panicked → produced 5xx errors
2. FL proxies (old engine): Failed to process bot scores → defaulted to zero → broke logic in Access, rules, authentication, and security products

To make things even more confusing, Cloudflare’s status page (hosted externally) briefly went offline too, creating a misleading early hypothesis that the outage was a massive DDoS attack.

It wasn’t. It was configuration drift.

Why This Seemingly Small Bug Became a Big Internet Event

Config files have become “just another part of the deployment pipeline,” especially in cloud platforms where machine-generated metadata drives features. But Cloudflare’s outage shows:

• Config is not static
• Config can be corrupted
• Config needs validation just like code

Because this file was distributed globally across tens of thousands of Cloudflare servers, a single flawed generation step caused a worldwide issue within minutes.

Distributed systems amplify mistakes — both good ones and bad ones.

Key Engineering Lessons We Should All Learn

1. Never rely on hidden assumptions

Cloudflare’s proxy code assumed the feature file would never exceed a certain size. That “should never happen” moment is often the birthplace of outages.

Lesson: Add explicit limits, schema checks, and sanity validations to all config ingestion paths.

2. Configuration is part of your software supply chain

The feature file was generated, replicated, and consumed automatically — no human intervention. That makes it just as risky as code.

Lesson: Treat configuration pipelines as first-class citizens: test them, validate them, gate them.

3. Build for graceful degradation, not hard crashes

A single .unwrap() took down parts of the internet.

Lesson: Fail softly. If config is invalid, degrade safely, skip rules, or revert to defaults — don’t panic.

4. Feature flags and kill switches are essential

Cloudflare themselves acknowledged the need for a more robust kill-switch system in their follow-up plans.

Lesson: Every modern engineering team should have:

• global feature kill switches
• fast configuration rollback
• manual override options

5. Monitoring needs context, not just alarms

Many engineers watching the outage thought it was an external attack. Alerting didn’t tell them where the failure was coming from, just that everything was “down.”

Lesson: Monitoring should distinguish between:

• origin failure
• edge failure
• config failure
• auth failure
• internal propagation issues

Context reduces misdiagnosis.

6. Observability tools must be lightweight during crises

During recovery, Cloudflare reported that their debugging systems started consuming high CPU, which slowed other mission-critical services.

Lesson: Your troubleshooting tools must use minimal resources when the system is stressed.

7. Transparency helps the whole industry learn

Cloudflare’s detailed post-mortem is a model for engineering culture. Their openness helps engineers worldwide understand real failure modes in large-scale systems.

Lesson: Share your failures. They help others avoid the same mistakes.

The Bigger Picture: Why This Incident Matters

Cloudflare’s outage wasn’t just a story about a config file. It was a reminder of how fragile the modern internet can be:

• We automate everything
• We trust our pipelines
• We deploy continuously
• We assume schemas won’t change
• We rely on millions of machines making the same decision correctly

But when the assumptions underneath those systems break, failures propagate at machine speed.

For engineers, SREs, DevOps teams, and platform architects, this incident underscores a fundamental truth:

Your system is only as resilient as your least-validated assumption.

Final Thoughts

Cloudflare’s outage was a blip in the timeline of the internet, but the lessons are timeless.

Distributed systems will always fail, the question is how gracefully they fail, how quickly they recover, and how deeply the organization learns from the event.

Cloudflare’s transparency, analysis, and remediation steps set a strong example for engineering teams everywhere.

Update (2026): Stripe’s Accounts v2 API uses a unified Account with configurations (merchant, customer, recipient) so one identity can sell, pay your platform, and receive transfers without parallel Customer IDs. For a full guide in the same spirit as this article—but aligned to v2—see Stripe Connect on Accounts v2. Everything below remains the reference for Accounts v1 (Account.create, OAuth, Account Links as shown). V2 though is the recommended path for the new Stripe Connect Integrations.

Stripe offers a powerful ecosystem for building payment platforms, marketplaces, and financial applications. One of its most powerful products, Stripe Connect, allows you to facilitate payments between multiple parties while maintaining flexibility over branding, compliance, and user experience.

Choosing between Standard, Express, and Custom accounts is one of the most important architectural decisions you’ll make when designing a payment platform. This guide combines a step-by-step tutorial and deep technical analysis to help you understand and implement each type.

What Is Stripe Connect?

Stripe Connect is designed for platforms that process payments on behalf of others. It enables you to connect user accounts (called Connected Accounts) to your Platform Account so you can:

• Collect payments from customers
• Distribute payouts to vendors, lenders, or service providers
• Optionally collect platform fees

Your platform never needs to hold funds directly (unless you design it to), and Stripe handles the movement of money under the hood.

Connected Accounts Overview

Each connected account represents a business, seller, or user on your platform. Depending on how much control and responsibility you want, you’ll choose between:

1. Standard — Stripe handles almost everything.
2. Express — Stripe handles compliance, you handle limited UX.
3. Custom — You handle everything; Stripe is invisible to your users.

Architectural Overview

All three integration types share a common flow:

Customer → Your Platform → Connected Account → Bank Payout

However, the control points differ:

• With Standard, Stripe owns the UX and dashboard.
• With Express, Stripe hosts onboarding and dashboard, but your platform manages relationships.
• With Custom, your platform builds everything: UI, onboarding, payouts, and reporting.

Step-by-Step Implementation by Account Type

Let’s look at how to implement each account type with C#, Laravel (PHP) and Python examples and understand their implications.

Standard Accounts — Easiest Setup, Minimal Control

Concept

Standard accounts are the simplest to integrate. Users connect their own existing Stripe accounts using OAuth. Stripe handles compliance, payouts, and reporting. Your platform receives a small application fee from each transaction.

Use Case

Ideal for marketplaces or SaaS platforms where users already have Stripe accounts and prefer to manage their own dashboards.

Implementation

C#

var accountLink = await stripe.AccountLinks.CreateAsync(new AccountLinkCreateOptions {
    Account = connectedAccountId,
    RefreshUrl = "https://your-platform.com/reauth",
    ReturnUrl = "https://your-platform.com/success",
    Type = "account_onboarding",
});

Laravel (PHP)

$accountLink = \Stripe\AccountLink::create([
    'account' => $connectedAccountId,
    'refresh_url' => route('stripe.reauth'),
    'return_url' => route('stripe.success'),
    'type' => 'account_onboarding',
]);

Python

import stripe
stripe.api_key = "sk_test_..."

account_link = stripe.AccountLink.create(
    account=connected_account_id,
    refresh_url="https://your-platform.com/reauth",
    return_url="https://your-platform.com/success",
    type="account_onboarding"
)

Pros and Cons

Pros

• Minimal setup
• Stripe handles everything (KYC, compliance, payouts)
• Reduced liability

Cons

• Limited branding control
• Users must leave your platform to manage payments
• Harder to create a unified experience

Express Accounts — Fast Onboarding, Shared Control

Concept

Express accounts strike a balance between simplicity and control. You manage account creation and linking, but Stripe handles onboarding and provides a lightweight dashboard for your users.

Use Case

Perfect for gig or service platforms (like driver or freelancer apps) where quick onboarding and basic payout visibility are key.

Implementation

C#

var account = await stripe.Accounts.CreateAsync(new AccountCreateOptions {
    Type = "express",
    Country = "US",
    Email = "user@example.com"
});

var link = await stripe.AccountLinks.CreateAsync(new AccountLinkCreateOptions {
    Account = account.Id,
    RefreshUrl = "https://yourapp.com/reauth",
    ReturnUrl = "https://yourapp.com/complete",
    Type = "account_onboarding"
});

Laravel (PHP)

$account = \Stripe\Account::create([
    'type' => 'express',
    'country' => 'US',
    'email' => $request->input('email'),
]);

$link = \Stripe\AccountLink::create([
    'account' => $account->id,
    'refresh_url' => route('stripe.reauth'),
    'return_url' => route('stripe.success'),
    'type' => 'account_onboarding',
]);

Python

account = stripe.Account.create(
    type="express",
    country="US",
    email="user@example.com"
)

link = stripe.AccountLink.create(
    account=account.id,
    refresh_url="https://yourapp.com/reauth",
    return_url="https://yourapp.com/complete",
    type="account_onboarding"
)

Pros and Cons

Pros

• Faster onboarding
• Stripe handles compliance and verification
• Users get access to payout history via the Express dashboard

Cons

• Limited customization
• Stripe branding remains visible
• Less flexibility over reporting or custom payout logic

Custom Accounts — Full Control, Full Responsibility

Concept

Custom accounts are designed for white-labeled platforms. Your app controls everything: onboarding, KYC collection, balances, and payouts. Stripe is completely invisible to the end user.

Use Case

Ideal for fintech, embedded finance, lending, or any system that requires deep integration and custom user experiences.

Implementation

C#

var accountOptions = new AccountCreateOptions {
    Type = "custom",
    Country = "US",
    Email = data.Email.ToLower(),
    Capabilities = new AccountCapabilitiesOptions {
        CardPayments = new AccountCapabilitiesCardPaymentsOptions { Requested = true },
        Transfers = new AccountCapabilitiesTransfersOptions { Requested = true }
    },
    TosAcceptance = new AccountTosAcceptanceOptions {
        Date = DateTimeOffset.UtcNow.ToUnixTimeSeconds(),
        Ip = httpContext.Connection.RemoteIpAddress.ToString()
    }
};

var account = await stripe.Accounts.CreateAsync(accountOptions);

Laravel (PHP)

$account = \Stripe\Account::create([
    'type' => 'custom',
    'country' => 'US',
    'email' => strtolower($data['email']),
    'capabilities' => [
        'card_payments' => ['requested' => true],
        'transfers' => ['requested' => true],
    ],
    'tos_acceptance' => [
        'date' => time(),
        'ip' => request()->ip(),
    ],
]);

Python

account = stripe.Account.create(
    type="custom",
    country="US",
    email=data["email"].lower(),
    capabilities={
        "card_payments": {"requested": True},
        "transfers": {"requested": True},
    },
    tos_acceptance={
        "date": int(time.time()),
        "ip": request.remote_addr,
    }
)

Fund Flow Example

+--------------------------------------+
|             Platform (You)           |
|    No direct money handling          |
+--------------------+-----------------+
                     |
                     ▼
          Create & Manage Connected Accounts
                     |
   +--------------------------------------+
   |     Connected Account (Business)     |
   |  💵 Has Stripe balance, bank info    |
   |  🧾 Disburses and collects funds     |
   +--------------------------------------+

Money Movement

• IN: Customer → Connected Account (via ACH/Card)
• OUT: Connected Account → Bank (payouts)
• Platform orchestrates, Stripe moves the money

Checking Balances

Python

balance = stripe.Balance.retrieve(stripe_account=account_id)

Laravel (PHP)

$balance = \Stripe\Balance::retrieve([], ['stripe_account' => $accountId]);

C#

var balance = await stripe.Balance.GetAsync(
    new BalanceGetOptions(),
    new RequestOptions { StripeAccount = accountId }
);

Compliance Responsibilities

Tip: For Custom accounts, implement Stripe Identity, webhooks, and Stripe Radar to automate verification and fraud detection.

Choosing the Right Integration Type

Strategic Considerations

• Time-to-market: Standard < Express < Custom
• Compliance load: Stripe-heavy → Custom-heavy
• Branding control: Minimal → Full
• Revenue potential: Low (Standard) → High (Custom)

Final Thoughts

Every Stripe Connect integration represents a trade-off between control, compliance, and complexity:

• Standard — easiest to deploy, lowest control
• Express — balanced control and simplicity
• Custom — ultimate flexibility with greater responsibility

If your goal is speed, start with Express. If your goal is brand control and scalability, build with Custom.

Either way, Stripe Connect gives you a future-proof foundation for managing payments, onboarding users, and creating rich financial experiences all through powerful APIs available across languages.

In this final part of our testing series, we’ll:

• Understand the Testing Pyramid model.
• Explore how to balance different types of tests.
• Learn how to integrate testing into CI/CD pipelines.
• Review best practices for building a sustainable testing culture.

The Testing Pyramid Explained

The Testing Pyramid (popularized by Mike Cohn) is a metaphor for structuring automated tests:

        ▲
        |   End-to-End (fewest, slowest)
        |
        |   Integration (moderate amount, balanced)
        |
        |   Unit Tests (largest base, fastest)
        ▼

1. Unit Tests (Foundation)

• Fast, cheap, and precise.
• Cover small pieces of logic in isolation.
• Should make up 60–70% of your automated test suite.

Example: Testing a function that calculates interest rates.

2. Integration Tests (Middle Layer)

• Validate how components interact.
• Cover database queries, API requests, or service orchestration.
• Should make up 20–30% of your test suite.

Example: Ensuring your API endpoint correctly fetches data from the database and formats the response.

3. End-to-End Tests (Top)

• Simulate real user flows across the full stack.
• Catch bugs unit or integration tests can’t.
• Should make up 10–15% of your suite (because they’re slow and expensive).

Example: A test where a user logs in, adds a product to their cart, and completes checkout.

Integrating Tests into CI/CD Pipelines

Automated tests are only valuable if they’re run consistently. Modern software teams embed them into CI/CD pipelines.

Step 1: Run Unit Tests Early

• Execute on every commit or pull request.
• Fail fast: block merges if unit tests fail.

Step 2: Run Integration Tests on Build/Deploy

• Run after the app compiles successfully.
• Can use a containerized test environment with mock or staging databases.

Step 3: Run End-to-End Tests on Staging

• Triggered before production deployment.
• Some teams run smoke E2E tests in production (carefully) to ensure critical flows still work.

Example CI/CD Flow (GitHub Actions / GitLab / Jenkins):

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Install dependencies
        run: npm install
      - name: Run unit tests
        run: npm test -- --unit
      - name: Run integration tests
        run: npm test -- --integration
      - name: Run e2e tests
        run: npm run test:e2e

Best Practices for a Balanced Testing Strategy

1. Follow the Pyramid, Not the Ice Cream Cone

• Too many E2E tests = slow, brittle pipeline.
• Too few unit tests = shaky foundation.
• Balance is key.

2. Use Test Doubles Wisely

• Mocks/Stubs in unit tests to isolate dependencies.
• Minimal mocking in integration tests — rely on real services where possible.

3. Make Tests Deterministic

• Tests should always produce the same result.
• Avoid flaky tests (caused by race conditions, timeouts, or external dependencies).

4. Keep Tests Fast

• Aim for seconds, not minutes.
• Developers won’t run slow tests locally.

5. Automate Everything in CI/CD

• Manual testing has its place, but regression checks must be automated.
• CI/CD pipelines should be your safety net before production.

6. Monitor and Improve Test Coverage

• Coverage isn’t everything, but low coverage usually indicates gaps.
• Focus on critical paths rather than chasing 100%.

7. Treat Tests as Code

• Tests should be maintainable, reviewed, and refactored like production code.
• Avoid “test rot” where tests become outdated or ignored.

Final Thoughts

The journey from unit → integration → E2E tests gives you confidence at every level of your system.

• Unit tests keep your building blocks solid.
• Integration tests ensure the blocks fit together.
• E2E tests confirm the entire structure works as intended.

By following the Testing Pyramid, integrating tests into CI/CD pipelines, and practicing discipline in writing effective tests, you’ll achieve:

• Faster feedback loops.
• Fewer production bugs.
• Higher developer confidence.
• Happier end-users.

Testing isn’t just about catching bugs, it’s about building trustworthy software at scale.

Next Steps for You:

• Audit your current test suite.
• See if you’re over-relying on one type of test.
• Gradually reshape your suite into a pyramid, not an ice cream cone.

E2E testing is about validating the system as a whole, covering everything from frontend to backend, database, APIs, and sometimes even external services.

What Are E2E Tests?

E2E tests replicate user behavior and verify that the system responds correctly. Think of them as automated users interacting with your app.

Example workflows tested in E2E:

• User logs in → Dashboard loads correctly.
• User creates a record → It’s stored in DB → Appears in the UI.
• Checkout flow in an e-commerce app → Product added → Payment processed → Order confirmed.

Why E2E Tests Matter

• Validate Real Workflows → Ensure the system behaves like users expect.
• Catch System-Wide Failures → Config issues, routing errors, DB schema mismatches.
• Test Critical Paths → Login, payments, onboarding, search, etc.
• Provide Business Confidence → Stakeholders see features working as intended.

Characteristics of E2E Tests

• High Coverage – Test across the full stack.
• Slowest of All Tests – Often run in CI/CD pipelines, not during active coding.
• Brittle – Small UI changes can break tests, so balance is key.
• Mimic Real User Behavior – Often browser-driven or API-driven.

Side-by-Side Examples

Scenario:

A user visits a web app, logs in, and sees their profile page with their name.

Python (Selenium + pytest)

# test_e2e_login.py
from selenium import webdriver
from selenium.webdriver.common.by import By
import pytest

@pytest.fixture
def driver():
    driver = webdriver.Chrome()
    yield driver
    driver.quit()

def test_login_flow(driver):
    driver.get("http://localhost:5000/login")

    driver.find_element(By.NAME, "username").send_keys("alice")
    driver.find_element(By.NAME, "password").send_keys("password123")
    driver.find_element(By.ID, "login-button").click()

    profile_name = driver.find_element(By.ID, "profile-name").text
    assert profile_name == "Alice"

C# (Selenium + xUnit)

using Xunit;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;

public class LoginE2ETests
{
    [Fact]
    public void Login_ShowsProfilePage()
    {
        using var driver = new ChromeDriver();
        driver.Navigate().GoToUrl("http://localhost:5000/login");

        driver.FindElement(By.Name("username")).SendKeys("alice");
        driver.FindElement(By.Name("password")).SendKeys("password123");
        driver.FindElement(By.Id("login-button")).Click();

        var profileName = driver.FindElement(By.Id("profile-name")).Text;
        Assert.Equal("Alice", profileName);
    }
}

TypeScript (Playwright / Cypress)

(Playwright example below — Cypress would be very similar)

// login.e2e.test.ts
import { test, expect } from "@playwright/test";

test("user can login and see profile page", async ({ page }) => {
  await page.goto("http://localhost:3000/login");

  await page.fill("input[name=username]", "alice");
  await page.fill("input[name=password]", "password123");
  await page.click("#login-button");

  const profileName = await page.textContent("#profile-name");
  expect(profileName).toBe("Alice");
});

PHP (Laravel Dusk for browser automation)

// tests/Browser/LoginTest.php
<?php

namespace Tests\Browser;

use Laravel\Dusk\Browser;
use Tests\DuskTestCase;

class LoginTest extends DuskTestCase
{
    public function testUserCanLoginAndSeeProfile()
    {
        $this->browse(function (Browser $browser) {
            $browser->visit('/login')
                    ->type('username', 'alice')
                    ->type('password', 'password123')
                    ->press('Login')
                    ->assertSee('Alice');
        });
    }
}

Comparison Table

Best Practices for Writing Effective E2E Tests

1. Test Critical User Journeys Only
   • Focus on login, checkout, payments, onboarding, etc.
   • Don’t waste time E2E testing every small feature.
2. Keep Tests Deterministic
   • Avoid flakiness by controlling test data.
   • Seed databases with known test records.
3. Use Test-Specific Environments
   • Run E2E tests in staging or CI environments.
   • Isolate from production data and services.
4. Clean Up After Tests
   • Ensure created records (users, orders) are rolled back or deleted.
5. Use IDs and Stable Selectors
   • Avoid brittle selectors like div:nth-child(3).
   • Use unique IDs or data-test attributes.
6. Parallelize and Optimize
   • Run tests in parallel (e.g., Playwright, Cypress).
   • Keep them minimal to avoid bloated CI runs.
7. Combine with Unit & Integration Tests
   • Don’t rely solely on E2E.

   • Use a test pyramid strategy:

     • 70% Unit Tests
     • 20% Integration Tests
     • 10% E2E Tests
8. Monitor and Review Regularly
   • E2E tests can get brittle.
   • Review and refactor when the UI or workflows change.

Key Takeaways

• E2E tests replicate the user’s journey and validate the full system.
• They are slower and more brittle, but they give the highest level of confidence.
• Use them sparingly for critical paths (login, payments, core workflows).

• Balance your test pyramid:

  • Many unit tests
  • Fewer integration tests
  • Very few but powerful E2E tests
• Choose the right tools for your stack (Selenium, Playwright, Cypress, Dusk).
• Follow best practices to keep tests reliable and maintainable.

This post is a practical, language-agnostic guide to integration testing, plus side-by-side, runnable patterns for Python, C# (.NET), TypeScript (Node/Express) and PHP (Laravel) so you can immediately apply the ideas in your stack.

What integration tests are (and are not)

Integration tests verify behavior across component boundaries, multiple classes, modules, services or infrastructure pieces that would not be exercised by a unit test.

They are not:

• A replacement for unit tests (they’re slower & coarser).
• Full UI-driven E2E tests (unless you intentionally include the UI).

They are good for:

• Verifying DB reads/writes via your data access layer.
• Testing service-to-service interactions.
• Ensuring message queue jobs and workers together produce expected state.
• Checking how your app handles external API payloads (with a mock or stub of that API).

Goals & tradeoffs

Goals

• Catch bugs that only appear when components are wired together.
• Validate API contracts inside your own system.
• Give more realistic coverage than unit tests.

Tradeoffs

• Slower than unit tests.
• Harder to make fully deterministic (external services, timing).
• Need careful setup/teardown to stay reliable.

Core patterns & recommendations

1. Use realistic but controlled dependencies

• Prefer a real database (or the same engine, e.g., PostgreSQL) rather than mocking DB calls.
• For external services (payment gateways, email providers), use service doubles: a local mock server, WireMock, or HTTP interceptors (nock, responses, Http::fake). Don’t call the live service in CI.

2. Isolate tests

• Run each test in a transaction and roll it back (if possible), or recreate schema between tests.
• Or give each test its own ephemeral database (unique DB name/per-worker) when running tests in parallel.

3. Keep tests focused

• Each integration test should exercise a meaningful interaction or flow (e.g., API -> DB, or API -> external-service-stub -> DB), not every possible path.

4. Seed deterministic test data

• Use builders/fixtures to create known state. Avoid random data unless seeded.

5. Manage long-running processes

• For queues/workers, either run workers synchronously in tests, use a fake queue, or spin up a test worker process in CI.

6. Use testcontainers or docker-compose in CI

• For close-to-production fidelity, use Testcontainers (or docker-compose) to provision real DBs and services in CI.

7. Avoid flaky tests

• No sleeps/time-based races. Use blocking signals, polling with timeouts, or deterministic stubs.

When to mock vs when to use real services

• Real DB: Prefer real DB engine (Postgres, MySQL). SQLite is OK for many cases but can mask engine-specific issues.
• External APIs: Mock in integration tests. Use contract testing (Pact) to keep mocked expectations in-sync.
• Caches/Queues: Use in-memory or test doubles unless you must validate the actual middleware behavior.

Observability: make debugging failing integration tests easy

• Emit structured logs during tests (include request IDs).
• Capture and print responses and DB state on failure.
• Keep helpful assertion messages.

Checklist: test lifecycle

1. Create test environment (DB, migrations applied).
2. Seed minimal deterministic data.
3. Execute action via real interfaces (HTTP client, direct call).
4. Assert state persisted, side effects happened (e.g., DB row created, message pushed, HTTP call stubbed).
5. Tear down (transaction rollback, truncate tables, drop DB).

Common pitfalls & fixes

• Flaky tests: avoid sleep-based waiting; use retry-with-timeout polling and assert deterministically.
• Slow setup: keep per-test setup minimal; use transactional rollback where possible.
• Parallel test collisions: give tests separate DBs or use unique table prefixes.
• “Works locally but fails in CI”: mirror CI environment locally using Docker/Testcontainers and run tests there.

Example integration tests (side-by-side)

Scenario used across examples:

A POST /users endpoint that creates a user record in the database and triggers an HTTP call to an external email service (welcome email). Integration test will create a user via HTTP, verify DB row exists, and verify the email call was made (mocked).

Python — FastAPI + SQLAlchemy + pytest + requests-mock

Notes: Use sqlite:///:memory: or a Testcontainers Postgres for higher fidelity in CI. requests-mock stubs outgoing HTTP calls.

# app.py (pseudo)
from fastapi import FastAPI, Depends
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

app = FastAPI()

# App factory to pass different DB URLs in tests
def create_app(db_url, email_service_url):
    engine = create_engine(db_url)
    Session = sessionmaker(bind=engine)
    # create tables...
    app.state.db = Session
    app.state.email_url = email_service_url

    @app.post("/users")
    def create_user(payload: dict):
        sess = app.state.db()
        user = User(name=payload["name"], email=payload["email"])
        sess.add(user); sess.commit()
        # Send welcome email via requests.post(app.state.email_url, json=...)
        return {"id": user.id}

    return app

# test_integration.py
import pytest
from fastapi.testclient import TestClient
import requests_mock
from app import create_app

@pytest.fixture
def client(tmp_path):
    # Use sqlite in-memory for speed or file DB for persistence across app
    app = create_app("sqlite:///:memory:", "http://email.test/send")
    client = TestClient(app)
    yield client

def test_create_user_and_send_email(client):
    with requests_mock.Mocker() as m:
        m.post("http://email.test/send", status_code=200, json={"ok": True})
        resp = client.post("/users", json={"name":"Alice","email":"a@example.com"})
        assert resp.status_code == 200
        user_id = resp.json()["id"]

        # Verify DB row exists (open a session)
        Session = client.app.state.db
        sess = Session()
        user = sess.query(User).filter_by(id=user_id).one_or_none()
        assert user is not None
        assert user.email == "a@example.com"

        # Verify external email call occurred
        assert m.called
        assert m.request_history[0].json() == {"to": "a@example.com", "template": "welcome"}

Tips

• For CI, replace sqlite with Testcontainers Postgres: create_app(postgres_url, ...).
• Use DB migrations in setup if using a real DB.

C# (.NET) — ASP.NET Core + WebApplicationFactory + InMemory DB / Testcontainer

Notes: Use WebApplicationFactory<TEntryPoint> to spin the app in tests and override service registrations for test doubles.

// In Startup.cs, app reads EmailService via IEmailService (HttpEmailService in prod)

public class TestEmailService : IEmailService {
    public List<EmailMessage> Sent = new();
    public Task SendAsync(EmailMessage msg) { Sent.Add(msg); return Task.CompletedTask; }
}

// Integration test
public class UsersIntegrationTests : IClassFixture<WebApplicationFactory<Program>> {
    private readonly WebApplicationFactory<Program> _factory;

    public UsersIntegrationTests(WebApplicationFactory<Program> factory) {
        _factory = factory.WithWebHostBuilder(builder => {
            builder.ConfigureServices(services => {
                // Replace real DB with in-memory or Testcontainer; replace email service with TestEmailService
                services.AddSingleton<IEmailService, TestEmailService>();
                // Configure EF Core to use InMemoryDatabase or connection string from Testcontainers
            });
        });
    }

    [Fact]
    public async Task PostUsers_CreatesUser_And_SendsEmail() {
        var client = _factory.CreateClient();
        var content = new StringContent("{\"name\":\"Bob\",\"email\":\"b@ex.com\"}", Encoding.UTF8, "application/json");
        var resp = await client.PostAsync("/users", content);
        resp.EnsureSuccessStatusCode();

        // Verify DB: use scope to resolve DbContext
        using(var scope = _factory.Services.CreateScope()) {
            var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
            var user = db.Users.Single(u => u.Email == "b@ex.com");
            Assert.NotNull(user);
        }

        // Verify TestEmailService captured message
        var emailService = _factory.Services.GetRequiredService<IEmailService>() as TestEmailService;
        Assert.Single(emailService.Sent);
        Assert.Equal("welcome", emailService.Sent[0].Template);
    }
}

Tips

• For a real DB in CI, use Testcontainers .NET to spin up Postgres and set EF Core connection string.
• Overriding services avoids brittle HTTP stubbing, and keeps assertions in-process.

TypeScript (Node/Express) — supertest + sqlite in-memory + nock

Notes: supertest issues HTTP requests to your Express app instance. Use nock to intercept outgoing HTTP.

// app.ts (pseudo)
import express from "express";
import bodyParser from "body-parser";
import { initDb } from "./db";

export function createApp(dbPath: string) {
  const app = express();
  app.use(bodyParser.json());
  const db = initDb(dbPath); // e.g., sqlite3 in-memory or file
  app.post("/users", async (req, res) => {
    const { name, email } = req.body;
    const id = await db.run("INSERT INTO users(name,email) VALUES(?,?)", [name, email]);
    // call external email service via fetch/http client
    await fetch("http://email.test/send", { method: "POST", body: JSON.stringify({ to: email, template: "welcome" }) });
    res.json({ id });
  });
  return app;
}

// test/integration.test.ts
import request from "supertest";
import nock from "nock";
import { createApp } from "../app";
import { openDb, getUserById } from "../db";

describe("POST /users", () => {
  it("creates a user and calls email service", async () => {
    const app = createApp(":memory:");
    const email = nock("http://email.test")
      .post("/send", (body) => body.to === "c@ex.com" && body.template === "welcome")
      .reply(200, { ok: true });

    const res = await request(app)
      .post("/users")
      .send({ name: "Carol", email: "c@ex.com" })
      .expect(200);

    // verify DB
    const user = await getUserById(res.body.id);
    expect(user.email).toBe("c@ex.com");

    // verify external call was made
    expect(email.isDone()).toBe(true);
  });
});

Tips

• For complex schemas, use migrations in test setup or run a dedicated test DB with sqlite file per test.
• For Postgres in CI, spin up DB via docker-compose or Testcontainers Node.

PHP (Laravel) — HTTP tests + RefreshDatabase + Http::fake()

Notes: Laravel has excellent integration testing helpers. RefreshDatabase runs migrations and transacts where possible. Use Http::fake() to intercept external HTTP.

// routes/api.php
Route::post('/users', [UserController::class, 'store']);

// UserController->store uses User model and Http::post('http://email.test/send', ...);

// tests/Feature/CreateUserTest.php
<?php
namespace Tests\Feature;

use Illuminate\Foundation\Testing\RefreshDatabase;
use Illuminate\Support\Facades\Http;
use Tests\TestCase;
use App\Models\User;

class CreateUserTest extends TestCase
{
    use RefreshDatabase;

    public function test_create_user_and_send_welcome_email()
    {
        Http::fake([
            'email.test/*' => Http::response(['ok' => true], 200),
        ]);

        $response = $this->postJson('/api/users', [
            'name' => 'Dan',
            'email' => 'd@ex.com',
        ]);

        $response->assertStatus(200);

        // verify DB
        $this->assertDatabaseHas('users', ['email' => 'd@ex.com']);

        // assert that an outbound call was made
        Http::assertSent(function ($request) {
            return $request->url() == 'http://email.test/send' &&
                   $request['to'] == 'd@ex.com' &&
                   $request['template'] == 'welcome';
        });
    }
}

Tips

• Laravel’s RefreshDatabase will use in-memory sqlite if configured, otherwise migrate a test DB.
• Use Queue::fake() to test that jobs were dispatched without executing background workers.

Practical integration testing strategies

Use transactions for isolation

• Wrap each test in a DB transaction and roll back at the end. Works well when everything runs in the same DB connection.
• Caveat: some ORMs/connections (e.g., tests that spawn separate processes) might not share transaction visibility.

Use in-memory DBs for speed, but test on real DBs in CI

• SQLite in-memory is fast, but can behave differently (indexing, SQL dialect). Complement local tests with real-engine tests in CI using containers.

Stubbing external HTTP reliably

• Python: responses or requests-mock
• Node: nock
• C#: WireMock.Net or replace typed HttpClient with test handler
• PHP: Laravel Http::fake()

Testcontainers — real dependencies in CI

• Spin up a Postgres, Redis, or Kafka container for integration tests.
• Testcontainers exists for many ecosystems (Java, .NET, Node, Python wrappers).

Contract testing for cross-team APIs

• Use Pact or similar to generate contracts from consumer tests and verify provider compliance in provider CI. This avoids brittle mocks and catches breaking API changes.

Background jobs & queues

• Either run job handlers inline (synchronously) in tests, use fake queues that record enqueued messages, or run a worker process in CI that reads from test queue.

How many integration tests should you write?

No fixed number. Aim for:

• Unit tests: many (business logic)
• Integration tests: enough to cover critical integration points (DB persistence, payment flows, auth)
• E2E tests: few (critical user paths)

A practical rule: write integration tests for each major DB operation and for the essential external integrations.

Debugging failing integration tests

• Print request/response bodies and DB rows on failure.
• Capture network traffic (or enable higher logging).
• Reproduce the failing test locally with the same CI container setup (Testcontainers makes that easy).
• If a test is flaky, add instrumentation and increase visibility; temporary retries mask real issues.

Sample integration test checklist

Before merging an integration test into CI:

• Test uses deterministic data.
• DB schema/migrations run and are applied in setup.
• External dependencies are stubbed or provided by test containers.
• Test cleans up (transaction rollback or truncation).
• No sleeps or time-based races.
• Test is focused on behavior, not implementation.

Wrapping up & next steps

Integration tests reduce the mismatch between isolated units and the full system. They give higher confidence than unit tests while being cheaper and faster than full E2E tests. When designed well they catch boundary problems early and make refactoring safer.

Next post in the series: End-to-End (E2E) Testing in Depth — we’ll cover realistic end-to-end strategies, UI-driving vs API-only E2E, test environments, flaky UI tests, and how to design low-maintenance high-value E2E checks.

Happy testing!

This is a continuation of the the Unit, Integration, and End-to-End Tests: Building Confidence in Your Software series. In case you missed it, be sure to check out the previous posts for a solid foundation. This guide goes deep into what makes a great unit test, how to structure them, how to avoid common pitfalls, and how to design code that’s easy to unit test.

What is a “Unit” (really)?

A unit is the smallest piece of behavior you care about verifying typically a function, method, or class. The unit:

• Has no external I/O (no network, DB, filesystem, clock, randomness).
• Has clear inputs and outputs (return value or state change).
• Can be run thousands of times deterministically and quickly.

Heuristic: if your test needs network access, sleeps, or a DB, it’s probably not a unit test.

The Goal of Unit Tests

• Prevent regressions close to the source of truth.
• Document behavior by example.
• Enable refactoring with confidence.
• Encourage good design (loose coupling, small interfaces).

Qualities of Great Unit Tests (F.I.R.S.T.)

• Fast – run in milliseconds; okay to run on every save/commit.
• Isolated – no shared state; independent from other tests.
• Repeatable – same result every run; no flakiness.
• Self-validating – clear pass/fail without manual inspection.
• Timely – written close to when the code is written (test-first or test-soon).

Anatomy of a Readable Test

Use Arrange–Act–Assert (AAA) or Given–When–Then. Keep one behavior per test.

test "calculate_total when valid items then returns sum minus discounts" {
  // Arrange
  items = [10, 20, 30]
  discount = 0.1
  sut = PriceCalculator() // SUT = System/Subject Under Test

  // Act
  total = sut.calculate_total(items, discount)

  // Assert
  assert_equal(54, total) // (10+20+30)=60; 10% off => 54
}

Naming patterns that scale

• MethodName_WhenCondition_ShouldExpectedOutcome
• Given_State_When_Action_Then_Outcome

Good names save hours of log-digging later.

What to Test (and What Not to)

Do test

• Business rules and edge cases (boundaries, empties, nulls, negatives, overflows).
• Error handling (exceptions, validation messages).
• Idempotency and invariants (calling twice has same effect).
• Serialization/parsing for your own formats.

Don’t test

• Framework code you don’t control.
• Trivial getters/setters (unless there’s logic).
• UI layout or CSS (move to integration/E2E if needed).

Test Data Without Pain

Messy data setup is the #1 cause of unreadable tests. Use builders.

order = OrderBuilder()
          .with_item("book", 20)
          .with_item("pen", 5)
          .with_discount(0.1)
          .build()

Patterns:

• Test Data Builder: fluent object construction.
• Mother Object: canonical “valid” objects you tweak per test.
• Parameterized tests: table-driven cases for variations.

Test Doubles: Choose the Right Tool

Not everything should be “real” in a unit test. Replace collaborators with test doubles:

Guideline: Prefer stubs/spies/fakes. Use mocks when you truly care about the interaction contract (e.g., exactly one call, in a specific order). Over-mocking couples tests to implementation details.

Taming Non-Determinism (Time, Randomness, Concurrency, I/O)

Flaky unit tests usually leak non-determinism. Isolate it behind interfaces:

Time

Introduce a Clock port.

interface Clock { now(): Instant }
class SystemClock implements Clock { now() = system_now() }
class FixedClock(t) implements Clock { now() = t }

Inject FixedClock in tests.

Randomness

Wrap the RNG:

interface RandomGen { next(): int }
class SeededRandom(seed) implements RandomGen { ... }
class FixedRandom(sequence) implements RandomGen { next() = sequence.pop() }

Concurrency

Avoid real threads in unit tests. Extract logic to pure functions; test scheduling via fake executors or synchronous dispatchers.

I/O

Abstract with ports/adapters (Repository, HttpClient, FileStore). Use in-memory fakes.

Assertions that Pull Their Weight

• Prefer specific asserts: assert_equal(54, total) over assert_true(total < 60).
• Provide custom messages: assert_equal(54, total, "10% discount not applied").
• Assert one behavior per test (multiple asserts okay if they describe one behavior).

Property-Based Tests (when examples aren’t enough)

Beyond example tests, check properties that must always hold.

Properties for calculate_total(items, d):

• Non-negativity: result ≥ 0
• Monotonicity: adding an item never decreases total
• Discount bounds: with 0 ≤ d ≤ 1, total ≤ sum(items)

property "adding item increases total" {
  for_all(item_price > 0, items >= []) {
    before = calc(items, d=0)
    after  = calc(items + [item_price], d=0)
    assert_true(after >= before)
  }
}

Use property tests to catch edge cases humans miss.

Structuring Your Test Suite

• Mirror production structure: src/price/calculator.* → test/price/calculator_test.*
• One SUT per file where possible.
• Common helpers in test_support/ (builders, fakes, assertions).
• Keep fixtures local unless truly shared.

Coverage: Useful, but Not the Goal

• Track line/statement and branch coverage to find blind spots.
• Don’t chase 100%. Aim for meaningful coverage of business logic and risk areas.
• Complement with mutation testing if available (ensures tests can detect real changes).

Test-Driven Development (TDD): Micro-cycles that Improve Design

Red → Green → Refactor:

1. Red: write a failing test that expresses desired behavior.
2. Green: implement the simplest code to pass it.
3. Refactor: improve design with tests staying green.

Even if you don’t TDD all the time, using short cycles on complex logic reduces waste and over-engineering.

Common Unit Test Smells (and Fixes)

• Brittle tests (fail after harmless refactors) → Assert on behavior, not internal calls/ordering (avoid over-mocking).
• Mega setups (50-line arrange blocks) → Introduce builders and sensible defaults.
• Hidden dependencies (time, singletons) → Add interfaces; inject Clock/Random/Config.
• Flaky tests (sometimes fail) → Remove sleeps; use fixed clocks, fakes, and synchronous executors.
• Logic in tests (ifs/loops) → Replace with parameterized tests or test data tables.

Worked Example (End-to-End Unit Test Thought Process)

Requirement: “Calculate order total with per-item prices, optional percentage discount, and tax applied after discount.”

Rules:

• Subtotal = sum(prices)
• Discounted = subtotal × (1 − discount) where 0 ≤ discount ≤ 1
• Total = round_to_cents(discounted × (1 + taxRate))

Example tests

Happy path

test "applies 10% discount then 16% tax" {
  sut = PriceCalculator(taxRate=0.16)
  total = sut.total([100, 50], discount=0.10)
  // subtotal = 150; after discount = 135; with tax = 156.6
  assert_equal(156.60, total)
}

Edge cases

test "empty items yields 0" {
  assert_equal(0.00, PriceCalculator(0.16).total([], discount=0))
}

test "discount is clamped between 0 and 1" {
  assert_equal(116.00, PriceCalculator(0.16).total([100], discount=2.0)) // treated as 1.0
  assert_equal(116.00, PriceCalculator(0.16).total([100], discount=-0.5)) // treated as 0.0
}

Rounding behavior

test "rounds to nearest cent (bankers or half-up as spec'd)" {
  // define and lock rounding policy; assert explicit expected cents
}

Property

property "adding an item never decreases total when discount fixed" { ... }

Note how tests pin down rounding, clamping, and the order of operations—classic sources of real-world bugs.

Design for Testability (so unit tests are easy)

• Dependency Injection: pass collaborators (Clock, Repo) via constructor/params.
• Pure Functions: push logic into pure units; keep side effects at the edges.
• Small Interfaces: program to ports; adapters do I/O.
• Single Responsibility: smaller units are easier to test and reason about.

Performance: Keep Tests Lightning-Fast

• Avoid costly setup; prefer in-memory collaborators.
• No sleeps/timeouts; fake the clock/scheduler.
• Run unit tests in a watch mode locally; keep CI under seconds for this suite.

When to Delete or Rewrite Tests

• Requirements changed → Update tests to reflect new truth.
• Test asserts an implementation detail → Rewrite to assert behavior.
• Chronic flakiness → Fix root cause or remove; flaky tests destroy trust.

A Minimal Template You Can Reuse

suite PriceCalculatorTests:
  setup:
    tax = 0.16
    sut = PriceCalculator(taxRate=tax)

  test "returns zero for empty items":
    expect sut.total([], discount=0) == 0.00

  test "applies discount before tax":
    expect sut.total([100], discount=0.10) == 104.40

  test "clamps invalid discounts":
    expect sut.total([100], discount=2.0) == 116.00

  test "non-negativity property":
    for_all item_lists:
      expect sut.total(item_lists, discount=0) >= 0

Checklist: Before You Commit

• Test name reads like a spec (explains when/then).
• Only one behavior under test.
• No hidden I/O/time/randomness.
• Clear, specific assertions (with messages).
• Fast (ms), repeatable, independent.
• Data setup is minimal and readable (builder/fixtures).

Side-by-Side Examples

We’ll use a simple scenario:

Function to calculate the discounted price given an original price and discount percentage.

Python (pytest / unittest)

# discount.py
def apply_discount(price: float, discount: float) -> float:
    if discount < 0 or discount > 100:
        raise ValueError("Discount must be between 0 and 100")
    return price - (price * discount / 100)

# test_discount.py
import pytest
from discount import apply_discount

def test_apply_discount_valid():
    assert apply_discount(100, 10) == 90

def test_apply_discount_zero_discount():
    assert apply_discount(100, 0) == 100

def test_apply_discount_invalid():
    with pytest.raises(ValueError):
        apply_discount(100, 150)

C# (xUnit)

// Discount.cs
public class Discount
{
    public static decimal ApplyDiscount(decimal price, decimal discount)
    {
        if (discount < 0 || discount > 100)
            throw new ArgumentException("Discount must be between 0 and 100");
      
        return price - (price * discount / 100);
    }
}

// DiscountTests.cs
using Xunit;

public class DiscountTests
{
    [Fact]
    public void ApplyDiscount_ValidDiscount_ReturnsDiscountedPrice()
    {
        var result = Discount.ApplyDiscount(100, 10);
        Assert.Equal(90, result);
    }

    [Fact]
    public void ApplyDiscount_ZeroDiscount_ReturnsSamePrice()
    {
        var result = Discount.ApplyDiscount(100, 0);
        Assert.Equal(100, result);
    }

    [Fact]
    public void ApplyDiscount_InvalidDiscount_ThrowsException()
    {
        Assert.Throws<ArgumentException>(() => Discount.ApplyDiscount(100, 150));
    }
}

TypeScript (Jest)

// discount.ts
export function applyDiscount(price: number, discount: number): number {
  if (discount < 0 || discount > 100) {
    throw new Error("Discount must be between 0 and 100");
  }
  return price - (price * discount / 100);
}

// discount.test.ts
import { applyDiscount } from "./discount";

test("applyDiscount with valid discount", () => {
  expect(applyDiscount(100, 10)).toBe(90);
});

test("applyDiscount with zero discount", () => {
  expect(applyDiscount(100, 0)).toBe(100);
});

test("applyDiscount with invalid discount", () => {
  expect(() => applyDiscount(100, 150)).toThrow();
});

PHP (Laravel / PHPUnit)

// app/Services/DiscountService.php
<?php
namespace App\Services;

class DiscountService {
    public function applyDiscount(float $price, float $discount): float {
        if ($discount < 0 || $discount > 100) {
            throw new \InvalidArgumentException("Discount must be between 0 and 100");
        }
        return $price - ($price * $discount / 100);
    }
}

// tests/Unit/DiscountServiceTest.php
<?php
namespace Tests\Unit;

use App\Services\DiscountService;
use PHPUnit\Framework\TestCase;

class DiscountServiceTest extends TestCase
{
    public function testApplyDiscountValid() {
        $service = new DiscountService();
        $this->assertEquals(90, $service->applyDiscount(100, 10));
    }

    public function testApplyDiscountZero() {
        $service = new DiscountService();
        $this->assertEquals(100, $service->applyDiscount(100, 0));
    }

    public function testApplyDiscountInvalid() {
        $this->expectException(\InvalidArgumentException::class);
        $service = new DiscountService();
        $service->applyDiscount(100, 150);
    }
}

Comparison Table

Key Takeaways

• Unit tests are your first safety net: they guarantee that individual building blocks work correctly.
• They should be small, isolated, and run fast.
• Mocks and stubs are often used when dependencies exist.
• Every language has its idiomatic testing framework, but the philosophy is the same.

What’s Next in the Series

Up next: Integration Testing in Depth — choosing boundaries, taming real dependencies, keeping tests reliable without turning them into E2E. Stay tuned!

In the world of software testing, three types of tests are most commonly discussed: unit tests, integration tests, and end-to-end (E2E) tests. Each serves a distinct purpose, each has strengths and trade-offs, and together they form the foundation of a reliable testing strategy.

This article provides a comprehensive, language-agnostic breakdown of these three pillars of testing, highlighting their importance, differences, and best practices.

1. Unit Tests: Validating the Smallest Pieces

Definition Unit tests focus on testing the smallest units of code in isolation, typically functions, methods, or classes. The goal is to verify that a specific piece of logic behaves exactly as intended.

Why they matter

• They provide fast feedback since they run quickly.
• They serve as documentation for how a function or module is supposed to work.
• They help catch regressions early, before code reaches higher environments.

Example Scenario Imagine you’re building a shopping cart system. A unit test might check that:

• Adding an item updates the cart total correctly.
• Removing an item decreases the count.
• Discount calculations apply properly.

This test doesn’t care about the database, the API, or the user interface it just cares about whether your calculateTotal(cartItems) function works.

Key Points for Good Unit Tests

• Keep them isolated—no databases, APIs, or file systems.
• Cover both happy paths and edge cases.
• Make them small and fast so they can run frequently.

2. Integration Tests: Verifying Modules Work Together

Definition Integration tests focus on testing how different modules, services, or components interact with each other. Unlike unit tests, they don’t isolate a single function they simulate realistic flows between parts of the system.

Why they matter

• Many bugs don’t occur in isolation but at the boundaries where systems communicate.
• They help ensure that your code modules, services, or APIs play well together.
• They give a higher level of confidence than unit tests but at a higher cost (slower, more complex).

Example Scenario In the shopping cart system, an integration test might check that:

• When an item is added to the cart, it’s saved in the database.
• The updated cart total is correctly retrieved via the API.
• A discount applied in the service layer reflects in the final invoice.

This test involves the database, service logic, and possibly the API layer.

Key Points for Good Integration Tests

• Focus on real-world workflows, not individual functions.
• Use test doubles (e.g., mocks, stubs, in-memory databases) where necessary to keep them manageable.
• Balance depth—don’t turn every integration test into a full E2E test.

3. End-to-End (E2E) Tests: Testing Like a User

Definition End-to-End tests validate the entire system from the user’s perspective. They simulate how a real user would interact with your application, covering everything from the front-end UI to the backend services and the database.

Why they matter

• They catch issues that unit or integration tests miss.
• They ensure the entire flow works in production-like conditions.
• They give the highest level of confidence before release.

Example Scenario For the shopping cart system, an E2E test might:

1. Open the application in a browser.
2. Log in as a user.
3. Add an item to the cart.
4. Apply a discount code.
5. Proceed to checkout and ensure the final invoice matches expectations.

This test validates everything: frontend, backend, authentication, database, and even third-party services.

Key Points for Good E2E Tests

• Keep them focused on critical user flows (checkout, login, payments).
• Minimize their number since they are slow and costly to maintain.
• Automate them in CI/CD but run them strategically (e.g., nightly builds, pre-release checks).

The Testing Pyramid: How They Work Together

Think of these tests as layers in a pyramid:

1. Unit tests form the base (most numerous, fastest).
2. Integration tests form the middle (fewer, slower).
3. E2E tests form the top (the least, but most comprehensive).

This balance ensures:

• You catch bugs early with unit tests.
• You validate real-world interactions with integration tests.
• You simulate user behavior with E2E tests.

Best Practices Across All Test Types

• Write clear, meaningful test names (e.g., should_apply_discount_when_valid_code_provided).
• Aim for coverage, not 100% coverage obsession—focus on meaningful tests.
• Automate tests in CI/CD pipelines for consistent feedback.
• Keep tests deterministic—they should pass or fail for the same reason every time.
• Continuously refactor tests just as you refactor code.

Final Thoughts

Good testing is about balance. Unit tests give you speed, integration tests give you realism, and E2E tests give you confidence from a user’s perspective. Together, they help you ship reliable software faster and with less stress.

This article is the first in a testing series where we’ll dive deeper into each type, exploring practical strategies, pitfalls to avoid, and real-world examples.

In the next article, we’ll break down Unit Testing in depth covering patterns, anti-patterns, and practical tips for writing effective unit tests.

In this article, we’ll explore how to build scalable backend systems using Python, C#, PHP, and Dart . We’ll look at architectural principles, performance tips, and practical examples in each language.

What Do We Mean by “Scalable Backend”?

A scalable backend system is one that can handle increasing load (users, requests, or data) without major rewrites or performance bottlenecks. Scalability comes in two main flavors:

• Vertical Scaling (Scale Up): Adding more CPU, RAM, or resources to a single server.
• Horizontal Scaling (Scale Out): Adding more servers or instances to distribute the load.

The best backends are built with scalability in mind from day one : modular, stateless, well-monitored, and optimized for both growth and resilience.

Core Principles of Scalable Backend Design

No matter the language, scalable systems share some common patterns:

1. Stateless Services → Each request should be independent. Store session data in Redis or a DB, not in memory.
2. Microservices Architecture → Break a monolithic app into smaller, independent services.
3. Load Balancing → Distribute requests evenly across multiple servers (NGINX, HAProxy, AWS ELB).
4. Caching → Use Redis, Memcached, or CDNs to reduce database load.
5. Database Scalability → Sharding, replication, and read-write separation.
6. Asynchronous Processing → Offload long tasks to queues (RabbitMQ, Kafka, Celery, Hangfire).
7. Monitoring & Logging → Use Prometheus, ELK stack, or Grafana for insights.

Python: Flexibility and Rapid Development

Python is loved for its developer-friendly syntax and rich ecosystem . While not the fastest language, Python excels when paired with the right tools.

Python Stack for Scalable Backends

• Frameworks: Django (monolithic but battle-tested), FastAPI (modern and async-friendly), Flask (lightweight).
• Async Support: asyncio, uvicorn, gunicorn for concurrent request handling.
• Task Queues: Celery + Redis for background jobs.

Example: Async Endpoint with FastAPI

from fastapi import FastAPI
import httpx

app = FastAPI()

@app.get("/users")
async def get_users():
    async with httpx.AsyncClient() as client:
        response = await client.get("https://jsonplaceholder.typicode.com/users")
    return response.json()

This async API can handle thousands of concurrent requests without blocking.

C#: Enterprise-Grade Performance

C# with .NET Core is a powerhouse for scalable, enterprise-grade systems. It’s fast, strongly typed, and built for concurrency .

C# Stack for Scalability

• Framework: ASP.NET Core (cross-platform, high-performance).
• Async Support: Built-in async/await.
• Background Jobs: Hangfire, Azure Functions, or MassTransit.
• Deployment: Docker + Kubernetes, Azure App Service.

Example: Async API in ASP.NET Core

[ApiController]
[Route("api/[controller]")]
public class UsersController : ControllerBase
{
    private readonly HttpClient _client;

    public UsersController(HttpClient client)
    {
        _client = client;
    }

    [HttpGet]
    public async Task<IActionResult> GetUsers()
    {
        var response = await _client.GetStringAsync("https://jsonplaceholder.typicode.com/users");
        return Ok(response);
    }
}

With built-in dependency injection and async/await, ASP.NET Core scales smoothly in production.

PHP: Still Relevant and Scalable

Many dismiss PHP as “legacy,” but with modern frameworks and PHP 8 , it’s extremely fast and widely deployed.

PHP Stack for Scalability

• Frameworks: Laravel, Symfony.
• Async Support: Swoole or ReactPHP for non-blocking I/O.
• Queues: Laravel Horizon, RabbitMQ, Redis.
• Deployment: NGINX + PHP-FPM, Docker, or AWS Lambda (Bref).

Example: Scalable API with Laravel

Route::get('/users', function () {
    return Http::get("https://jsonplaceholder.typicode.com/users")->json();
});

Add Redis caching and Laravel Horizon to process jobs, and PHP apps scale to millions of requests per day .

Dart: The New Player for Backend Development

Dart is best known for Flutter mobile apps , but with Dart Frog and Shelf , it’s emerging as a backend option. Its async-first nature makes it perfect for scalable APIs.

Dart Stack for Scalability

• Frameworks: Shelf, Dart Frog.
• Async I/O: Built-in async/await.
• Deployment: Docker + Kubernetes, Firebase Functions.

Example: Simple Dart Shelf API

import 'dart:io';
import 'package:shelf/shelf.dart';
import 'package:shelf/shelf_io.dart' as io;

void main() async {
  var handler = const Pipeline()
      .addMiddleware(logRequests())
      .addHandler((Request req) {
        return Response.ok('Hello, scalable world!');
      });

  var server = await io.serve(handler, InternetAddress.anyIPv4, 8080);
  print('Server running on http://${server.address.host}:${server.port}');
}

Dart’s async model is similar to Node.js, but with strong typing and better performance consistency .

Comparing the Four Languages

Best Practices for Scaling Backends (Any Language)

1. Use APM tools (Datadog, New Relic) for performance monitoring.
2. Implement circuit breakers (Hystrix pattern) to prevent cascading failures.
3. Automate CI/CD pipelines for smooth deployments.
4. Prefer event-driven architecture for real-time scalability.
5. Benchmark with load testing tools (k6, JMeter, Locust).

Final Thoughts

Building a scalable backend isn’t about picking the “best” language—it’s about designing with scalability principles in mind.

• Python gives you flexibility and speed of development.
• C# delivers enterprise stability and blazing performance.
• PHP remains a battle-tested workhorse with a huge ecosystem.
• Dart offers exciting opportunities for mobile-first backends.

The key takeaway: Focus on architecture first, language second.