1. Liquid Victor : Media tracking and aggregation [used to assemble this presentation]
2. Prehensile Pony-Tail : A static site generator built in c#
3. TestHelperExtensions : A set of extension methods helpful when building unit tests
4. Conference Scheduler : A conference schedule optimizer
5. IntentBot : A microservices framework for creating conversational bots on top of Bot Framework
6. LiquidNun : Library of abstractions and implementations for loosely-coupled applications
7. Toastmasters Agenda : A c# library and website for generating agenda's for Toastmasters meetings
8. ProtoBuf Data Mapper : A c# library for mapping and transforming ProtoBuf messages

[ īdemˈpōt(ə)nce , ˈēdemˌpōt(ə)nce ]

The ability to execute a task an arbitrary number of times (>1) and have the resulting state of the system be the same as if the task was executed once.

An attempt to make more than 1 reliable change to the state of a system in a single invocation

• Reliable = Can reasonably be expected to occur ONCE^*
  • Irrespective of:
    • Reliability of the environment
    • Reliability of any dependencies
• Change = An ATOMIC modification
  • Database or Cache Update
  • Message Delivery
• Invocation = Automated Response to a Signal
  • User Action
  • Message Received

• Errors resulting from dual-writes can manifest in different ways
  • Duplicate Data or Actions
    • Duplicate Emails
    • Duplicate Billing
  • Missing Data or Actions
    • No email sent
    • No billing issued
  • Incomplete Data
    • Email sent but no record of it
• Supporting these systems can be a nightmare
  • Difficult to identify when errors occur
  • Difficult to remediate
  • Extremely difficult to replicate

Goal: Reliably update a Data Store and trigger an additional action

• A single atomic write that makes two updates
  • A change to the state stored in the DB
  • A command to execute a secondary process

• Advantages

  • Persist quickly
  • Captures every update
  • Uses familiar tech

• Disadvantages

  • Hard to scale
  • Harder to fanout than other patterns
  • Uses DB as a queue
  • Adds weight to the persistance (trx)
  • Still susceptible to errors on commit

Goal: Reliably update a Data Store and trigger an additional action

• A single atomic write that makes one update to the DB
• The DB log is used to trigger the secondary action

• Advantages

  • Persist quickly
  • Easy to scale
  • Easy to fanout

• Disadvantages

  • Not guaranteed to capture every update
  • Still susceptible to errors on commit

Goal: Reliably persist a message and trigger additional actions

• A single atomic write that makes one update to the message stream
• The stream is used to trigger all actions

• These patterns get very close to 4-9's
• Use Idempotence whenever possible
• Exactly-Once Delivery may be possible
  • Idempotent ingress (i.e. to Kafka)
  • Atomic updates of changes alongside queue offsets

Never "add on" inside an execution context

• 3-9's (unimportant)

  • 99.9% uptime
  • 1 outage second in 1000
  • ~ 43 min / month

• 4-9's (important)

  • 99.99% uptime
  • 1 outage second in 10,000
  • ~ 4.3 min / month

• 5-9's (critical)

  • 99.999% uptime
  • 1 outage second in 100,000
  • ~ 0.43 min (26 sec) / month

• At-Least Once

  • Every messages will be delivered 1+ times
  • The vast majority of message systems make this guarantee

• At-Most once

  • Every message will be delivered 0-1 times
  • i.e. Logging

• Exactly once

  • Every message will be delivered 1 and only 1 time
  • Very difficult and expensive
  • Limited usefulness while maintaining this guarantee
  • Combination of idempotent input and a downstream transaction