Pulse

A high-availability, distributed task scheduler featuring timed job execution, programmatic failure recovery, and an infrastructure-as-code deployment pipeline.

System Architecture

The scheduler is split into decoupled, role-based microservices to achieve fault-tolerance and horizontal scaling:

• Database (PostgreSQL): Manages the global scheduled task calendar.
• Scheduler: Polls the database for due tasks and dispatches execution triggers over the network.
• Message Broker (AWS SQS via Floci): Acts as the highly-available distribution layer for task dispatching.
• Worker Fleet: Distributed, containerized consumers that long-poll SQS, execute payloads, and report periodic heartbeats back to the DB.
• Reaper: Background cleaner that detects crashed workers via heartbeat timeouts and schedules automatic retries.
• Terraform Pipeline: Provisions the cloud-emulated queuing resources programmatically.

Tech Stack

• Go 1.26
• PostgreSQL
• AWS SDK for Go v2 & Terraform AWS Provider
• Floci (Local AWS Emulator)
• Docker & Docker Compose

How to Run

1. Launch the Local Cluster Topology

Boot up the core emulated cloud platform (PostgreSQL and Floci SQS) and scale up a distributed fleet of 3 worker nodes:

docker compose up --build --scale worker=3

2. Deploy Infrastructure via Terraform

In a separate terminal, execute the local cloud pipeline to provision SQS resources:

cd terraform
terraform init
terraform apply -auto-approve

3. Open the UI & Stress Test

• Open http://localhost:8080 in your web browser to monitor the system status.
• Execute the stress testing suite to queue 50 timed tasks and verify load balancing and recovery across your worker containers:

  ./stress.sh