Google Cloud Accelerated Platforms

This repository is a collection of accelerated platform best practices, reference architectures, example use cases, reference implementations, and various other assets on Google Cloud.

An accelerated platform utilizes specialized hardware components, or accelerators, such as GPUs (Graphics Processing Units) and TPUs (Tensor Processing Units), to significantly increase the speed of computationally intensive tasks. These tasks may include data analysis, machine learning, artificial intelligence, and graphics rendering. By offloading demanding workloads from traditional CPUs to dedicated hardware accelerators, which are capable of much faster parallel calculations, the platform optimizes high-performance computing.

Cloud Workstations (CWS)

Note

The Cloud Workstations (CWS) Platform is currently in beta and is still being actively developed.

The Cloud Workstations (CWS) Platform is a core, best practices, fully managed workstation environments built to meet the needs of security-sensitive enterprises. It enhances the security of workstation environments while accelerating onboarding and productivity.

• Cloud Workstations reference architecture
  • Cloud Workstations reference implementation

Google Kubernetes Engine (GKE)

GKE Base Platform

The GKE Base Platform is an implementation of a foundational platform built on GKE that incorporates best practices and provides a core environment optimized for running accelerated workloads. It offers a streamlined and efficient solution to leverage the benefits of GKE as the primary runtime.

Example implementations

• ComfyUI reference implementation

• Federated learning

• Inference reference architecture

  • Inference reference implementation
    • Online inference with GPUs
      • Online inference using Diffusers with GPUs on Google Kubernetes Engine (GKE)
      • Online inference using vLLM with GPUs on Google Kubernetes Engine (GKE)
    • Online inference with TPUs
      • Online inference using MaxDiffusion with TPUs on Google Kubernetes Engine (GKE)
      • Online inference using vLLM with TPUs on Google Kubernetes Engine (GKE)
    • Benchmarking Online inference performance on Google Kubernetes Engine (GKE)
    • Batch inference with GPUs
    • Offline batch inference with GPUs
    • Intelligent inference scheduling quickstart using llm-d
    • Intelligent inference scheduling using llm-d with GPU on Google Kubernetes Engine (GKE)

• Training reference architecture

  • Model fine tuning
    • Data processing
    • Data preparation
    • Fine tuning
    • Model evaluation

Guides

• LLM Inference Optimization: Achieving faster Pod Startup with Google Cloud Storage
• Optimizing GKE Workloads with Custom Compute Classes

[Deprecated] Playground AI/ML Platform on GKE

The Playground AI/ML Platform on GKE is a quick-start implementation of the platform that can be used to familiarize yourself with the GKE architecture and to get an understanding of various concepts covered in the use cases.

• GKE AI/ML Platform for enabling AI/ML Ops

Use cases

• Scalable and Distributed LLM Inference on GKE with vLLM
• Retrieval Augmented Generation (RAG) pipeline

Contributing

For more information about contributing to this repository, see CONTRIBUTING.