How to Turn Long-Running Python Simulations Into a Self-Service Platform

Transforming a long-running Python simulation into a multi-user, self-service platform shifts the primary challenge from compute to job control and infrastructure. The key architectural decision is to decouple job submission from execution using a queue, such as AWS SQS, to manage demand and handle back-pressure. This asynchronous approach allows for better state tracking, failure recovery, and isolation between runs, turning a manual process into a reusable service. However, this design introduces complexities like maintaining trustworthy job state across distributed components and ensuring idempotency due to at-least-once message delivery. The engineering effort focuses on creating a reliable system for queueing, state management, and reproducibility rather than just optimizing the simulation's code.