OpenInfra Summit 2022: Interoperable Hardware Acceleration for Serverless

Abstract

As Serverless computing dominates the SaaS market, the lack of support for hardware acceleration imposes an important limitation to AI/ML applications. To address this issue, we build vAccel, a framework that enables portable and secure hardware acceleration in multi-tenant environments.

vAccel decouples the user application from hardware-specific code: vAccel plugins hold the hardware-specific code implementing specific operations for a particular hardware accelerator; the vAccel runtime exposes these functions to user applications and maps them to their respective implementations.

vAccel applications migrate between heterogeneous hosts without code modifications or re-compilation. At the same time, user code never reaches the hardware accelerator: vAccel plugins are vendor-audited snippets that are offered to users as high-level functions (e.g. image classification). Moreover, vAccel employs an API remoting approach for exposing hardware acceleration inside virtual machines.