Treffer: Parikshan: Live Debugging of Production Systems in Isolation

Modern 24x7 SOA applications rely on short deployment cycles, and fast bug resolution to maintain their services. Hence, time-to-bug localization is extremely important for any SOA application. We present live debugging, a mechanism which allows debugging of production systems (run test-cases, debug, or profile, etc.) on-the-fly. We leverage user-space virtualization technology (OpenVZ/LXC), to launch containers cloned and migrated from running instances of an application, thereby having two containers: production (which provides the real output), and debug (for debugging). The debug container provides a sandbox environment for debugging without any perturbation to the production environment. Customized network-proxy agents replicate or replay network inputs from clients to both the production and debug-container, as well as safely discard all network output from the debug-container. We used our system, called Parikshan, to do live debugging on several real-world bugs, and effectively reduced debugging complexity and time.