Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions docs/content/en/docs-dev/plugin-development/01-introduction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
title: "Introduction"
weight: 1
description: >
Introduction to the PipeCD Pluggable Architecture and the goal of this book.
---

### About this Book

PipeCD is an open-source, GitOps-style Continuous Delivery platform. Recently, the PipeCD project introduced a new **Plugin Architecture** in its Alpha release to allow developers to flexibly extend PipeCD's deployment capabilities to support arbitrary platforms and tools.

In this book, we will learn how to build a custom PipeCD plugin from scratch by implementing a plugin named `file`. This plugin will allow PipeCD to manage and sync files on a local file system.

### Reference Links

Here are some useful reference links regarding PipeCD and its Plugin Architecture:

- [Official PipeCD Website](https://pipecd.dev/)
- [Official Plugin SDK for Go](https://pkg.go.dev/github.com/pipe-cd/piped-plugin-sdk-go)
- [Official Blog Post: Overview of the Plan for Pluggable PipeCD](https://pipecd.dev/blog/2024/11/28/overview-of-the-plan-for-pluginnable-pipecd/)
- [Japanese Article: Explaining the Plugin Architecture](https://zenn.dev/cadp/articles/pipecd-plugin-intro)

### Note on SDK Stability

This book is based on the PipeCD Plugin SDK version `v0.0.0-20250619080234-1ee9423d23c1` released during the Alpha stage. As pluggable architecture is continuously evolving, we recommend verifying Go interfaces, RPC definitions, and the `pipedv1` architecture against the latest default branch in the [`pipecd`](https://github.com/pipe-cd/pipecd) repository when implementing production-grade plugins.
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
title: "Plugin Features to Implement"
weight: 2
description: >
Determining the features and stages our custom plugin will provide.
---

First, let's decide what kind of features our custom plugin will provide.

In this book, we will implement a plugin that manages files on the local machine where `piped` is running. While `piped` is often run in containerized environments (such as Kubernetes), it can also be run directly on VMs or bare-metal machines. Practically, a local file-system plugin like this could be used to deploy configuration files onto a VM—taking over some of the tasks typically handled by tools like Chef or Ansible.

### Specific Features to Implement

Each PipeCD Application will have a designated target deployment directory. The plugin's job is to copy files located under the same directory as the `app.pipecd.yaml` configuration file from the Git repository directly into that target directory.

We will define two deployment stages for this plugin:

1. **`FILE_DIFF`**: This stage compares the files in the Git repository with the files actually deployed in the target directory, printing the differences to the deployment log. This is similar to running `terraform plan` or `kubectl diff`.
2. **`FILE_SYNC`**: This stage executes the actual sync operation by copying all files from the Git repository to the target directory and deleting any orphaned files in the target directory that do not exist in the Git repository.
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
title: "Technology Selection"
weight: 3
description: >
Selecting the language and development tools for our plugin.
---

Next, let's select our development language and tools.

When developing plugins for PipeCD, **Go** is the most natural choice because PipeCD officially provides the [piped-plugin-sdk-go](https://pkg.go.dev/github.com/pipe-cd/piped-plugin-sdk-go). Therefore, we will build our plugin using Go.

To keep things educational and focused on understanding the core SDK, we will use the standard library as much as possible alongside the official SDK, avoiding any third-party dependencies.

The version of the SDK used in this book is `v0.0.0-20250619080234-1ee9423d23c1`. Since pluggable architecture is continuously evolving, always check the latest official documentation when developing production-ready plugins.
18 changes: 18 additions & 0 deletions docs/content/en/docs-dev/plugin-development/04-project-init.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
title: "Initializing the Project"
weight: 4
description: >
Initializing the Git repository and Go module.
---

Let's begin by creating our Git repository and initializing a Go module.

Run the following commands in your terminal in a suitable working directory. Replace `<YOUR_USERNAME>` with your GitHub username or any appropriate identifier:

```console
$ git init pipecd-plugin-file
$ cd pipecd-plugin-file
$ go mod init github.com/<YOUR_USERNAME>/pipecd-plugin-file
```

Once you have initialized the project, make your initial git commit to save your progress. We highly recommend committing your work at small, logical milestones throughout this guide.
14 changes: 14 additions & 0 deletions docs/content/en/docs-dev/plugin-development/05-dependencies.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
title: "Adding Dependency Libraries"
weight: 5
description: >
Adding the official PipeCD plugin SDK to our project dependencies.
---

To enable autocompletion and type checking in your editor, add the official PipeCD Plugin SDK to your Go module dependencies by running:

```console
$ go get github.com/pipe-cd/piped-plugin-sdk-go
```

As mentioned earlier, we are focusing on building our understanding of the core SDK, so we will not be adding any other third-party dependencies.
20 changes: 20 additions & 0 deletions docs/content/en/docs-dev/plugin-development/06-plugin-types.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
title: "Understanding Plugin Types"
weight: 6
description: >
Deep dive into the types of plugins supported by the PipeCD SDK.
---

While PipeCD's core agent `Piped` interacts with plugins transparently, the official Go SDK categorizes plugins into distinct types to simplify development.

### 1. [StagePlugin](https://pkg.go.dev/github.com/pipe-cd/piped-plugin-sdk-go#StagePlugin)

A utility plugin that does not manage resources but provides utility stages to be executed within a deployment pipeline. An example is a plugin that provides a `WAIT` stage to pause the deployment pipeline for a defined duration.

### 2. [DeploymentPlugin](https://pkg.go.dev/github.com/pipe-cd/piped-plugin-sdk-go#DeploymentPlugin)

A plugin that manages actual target resources and performs state synchronization (syncing, diffing, rolling back). A Kubernetes plugin is a prime example.

In addition to core sync capabilities, a `DeploymentPlugin` can also implement the [LiveStatePlugin](https://pkg.go.dev/github.com/pipe-cd/piped-plugin-sdk-go#LiveStatePlugin) interface to report the live status of active resources on the target platform and render differences in the PipeCD Web UI.

When starting a plugin project, choose the type that fits your goals and implement its required interface. Since our `file` plugin manages local files as target resources, it will be implemented as a **`DeploymentPlugin`**.
43 changes: 43 additions & 0 deletions docs/content/en/docs-dev/plugin-development/07-first-steps.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
title: "First Step of Plugin Implementation"
weight: 7
description: >
Creating the main function and starting the gRPC server.
---

Let's begin writing our plugin code by implementing the `main` function.

Create a file named `main.go` and save the following code:

```go
package main

import (
"log"

sdk "github.com/pipe-cd/piped-plugin-sdk-go"
)

func main() {
plugin, err := sdk.NewPlugin[any, any, any]("0.0.1")
if err != nil {
log.Fatalln(err)
}

if err := plugin.Run(); err != nil {
log.Fatalln(err)
}
}
```

The type parameters passed to `sdk.NewPlugin` are temporary placeholders. By the time we finish implementing the plugin, Go's type inference will determine these automatically, but for now we must explicitly write them to compile.

### Note on Plugin Naming

You might have noticed that the string `file` (the name of our plugin) does not appear anywhere in this code.

Indeed, a plugin binary has no concept of its own name. The naming is entirely determined in the Piped configuration written by the user. A user could configure Piped to recognize this plugin binary as `file`, or `filesystem`, or any other name. The name itself is decoupled from the binary.

### Running this Code

At this point, if you attempt to run `go run main.go`, it will fail because `sdk.NewPlugin` returns an error if no plugin implementation is registered. In the next section, we will define a struct and satisfy the `DeploymentPlugin` interface.
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
title: "Satisfying the DeploymentPlugin Interface"
weight: 8
description: >
Analyzing the Go interface methods required to implement a DeploymentPlugin.
---

To register a plugin as a [DeploymentPlugin](https://pkg.go.dev/github.com/pipe-cd/piped-plugin-sdk-go#DeploymentPlugin) via `sdk.NewPlugin`, we must implement its interface methods.

First, let's understand the three type parameters required by `DeploymentPlugin`:

1. **`Config`**: The global configuration common to all instances of this plugin. This is defined in Piped's global configuration file.
2. **`DeployTargetConfig`**: The configuration unique to each deployment target. For example, in a Kubernetes plugin, this represents the target cluster credentials and context.
3. **`ApplicationConfigSpec`**: The configuration specified per application in `app.pipecd.yaml`. For example, a list of target manifest files or resource configurations.

### Interface Methods

To fully implement a `DeploymentPlugin` (including the embedded `StagePlugin` methods), we must define the following methods on our plugin struct:

- `FetchDefinedStages() []string`
- `DetermineVersions(context.Context, *Config, *DetermineVersionsInput[ApplicationConfigSpec]) (*DetermineVersionsResponse, error)`
- `DetermineStrategy(context.Context, *Config, *DetermineStrategyInput[ApplicationConfigSpec]) (*DetermineStrategyResponse, error)`
- `BuildPipelineSyncStages(context.Context, *Config, *BuildPipelineSyncStagesInput) (*BuildPipelineSyncStagesResponse, error)`
- `BuildQuickSyncStages(context.Context, *Config, *BuildQuickSyncStagesInput) (*BuildQuickSyncStagesResponse, error)`
- `ExecuteStage(context.Context, *Config, []*DeployTarget[DeployTargetConfig], *ExecuteStageInput[ApplicationConfigSpec]) (*ExecuteStageResponse, error)`

We will implement each of these methods step-by-step.
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
title: "Implementation: Defining Configuration Types"
weight: 9
description: >
Defining the configuration Go structs for our file plugin.
---

Our `file` plugin needs to allow users to specify a unique target directory path per application. While we could theoretically define this in a Deploy Target, requiring a separate Deploy Target for every single directory would be tedious for users. Therefore, we will define the target directory path within each application's configuration.

For future extensions—for instance, if we want this plugin to deploy files not just to the local machine but to remote hosts via SSH—the SSH connection details (host, keys, etc.) would be defined within the Deploy Target configuration, whereas the specific path on that host would remain in the Application configuration.

Add the following type definitions to `main.go` (or a separate file under `package main`):

```go
type (
// config represents the global plugin configuration.
// Since our file plugin does not require global settings, we leave it empty.
config struct{}

// deployTargetConfig represents the deployment target settings.
// Since our file plugin only interacts with the local machine, we leave it empty.
deployTargetConfig struct{}

// applicationConfig represents the configuration defined per application in app.pipecd.yaml.
// Here, we define the target directory path where the files should be synchronized.
applicationConfig struct {
// Path specifies the absolute path on the local file system where the files should be synced.
Path string `json:"path"`
}
)
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
title: "Implementation: Empty Implementation to Satisfy the Interface"
weight: 10
description: >
Scaffolding the plugin struct and satisfying the interface with stub methods.
---

Before implementing the logic for each method, let's establish a skeleton structure in `main.go`. This guarantees our method signatures match the expected interface and allows the project to compile.

Add the following code to `main.go` (or a separate file under `package main`):

```go
package main

import (
"context"

sdk "github.com/pipe-cd/piped-plugin-sdk-go/pkg/plugin/sdk"
)

// Ensure that plugin implements the sdk.DeploymentPlugin interface.
// If any methods are missing or have incorrect signatures, this will trigger a compile-time error.
var _ sdk.DeploymentPlugin[config, deployTargetConfig, applicationConfig] = plugin{}

// plugin provides the implementation of the DeploymentPlugin interface.
type plugin struct{}

// FetchDefinedStages returns the list of stages supported by this plugin.
func (plugin) FetchDefinedStages() []string {
panic("unimplemented")
}

// DetermineVersions determines the version of the resource being deployed.
func (plugin) DetermineVersions(ctx context.Context, cfg *config, input *sdk.DetermineVersionsInput[applicationConfig]) (*sdk.DetermineVersionsResponse, error) {
panic("unimplemented")
}

// DetermineStrategy decides whether to execute a Quick Sync or a Pipeline Sync.
func (plugin) DetermineStrategy(ctx context.Context, cfg *config, input *sdk.DetermineStrategyInput[applicationConfig]) (*sdk.DetermineStrategyResponse, error) {
panic("unimplemented")
}

// BuildPipelineSyncStages constructs the pipeline stages for a Pipeline Sync.
func (plugin) BuildPipelineSyncStages(ctx context.Context, cfg *config, input *sdk.BuildPipelineSyncStagesInput) (*sdk.BuildPipelineSyncStagesResponse, error) {
panic("unimplemented")
}

// BuildQuickSyncStages constructs the stages for a Quick Sync.
func (plugin) BuildQuickSyncStages(ctx context.Context, cfg *config, input *sdk.BuildQuickSyncStagesInput) (*sdk.BuildQuickSyncStagesResponse, error) {
panic("unimplemented")
}

// ExecuteStage runs the actual sync logic for a specific stage.
func (plugin) ExecuteStage(ctx context.Context, cfg *config, targets []*sdk.DeployTarget[deployTargetConfig], input *sdk.ExecuteStageInput[applicationConfig]) (*sdk.ExecuteStageResponse, error) {
panic("unimplemented")
}
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
title: "Implementation: FetchDefinedStages"
weight: 11
description: >
Implementing FetchDefinedStages to define custom deployment stages.
---

`FetchDefinedStages` returns the names of the stages supported by this plugin.

Our `file` plugin will support three stages: `FILE_DIFF`, `FILE_SYNC`, and `FILE_ROLLBACK`. Defining these stage names as constants is best practice so we can reuse them across other methods.

Update the `FetchDefinedStages` implementation and define the constants as follows:

```go
const (
stageDiff = "FILE_DIFF"
stageSync = "FILE_SYNC"
stageRollback = "FILE_ROLLBACK"
)

func (plugin) FetchDefinedStages() []string {
return []string{
stageDiff,
stageSync,
stageRollback,
}
}
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
title: "Implementation: DetermineVersions"
weight: 12
description: >
Implementing DetermineVersions to manage version tracking.
---

`DetermineVersions` returns the version of the resource being deployed.

Since our plugin synchronizes files from a Git repository, we will use the Git commit hash of the deployment source as our resource version.

The deployment source details are passed via `input.Request.DeploymentSource`. We will fetch `CommitHash` from it. We do not need to specify the artifact name or URL for this local plugin.

Update `DetermineVersions` as follows:

```go
func (plugin) DetermineVersions(_ context.Context, _ *config, input *sdk.DetermineVersionsInput[applicationConfig]) (*sdk.DetermineVersionsResponse, error) {
return &sdk.DetermineVersionsResponse{
Versions: []sdk.ArtifactVersion{{Version: input.Request.DeploymentSource.CommitHash}},
}, nil
}
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
title: "Implementation: DetermineStrategy"
weight: 13
description: >
Implementing DetermineStrategy to configure sync execution strategies.
---

`DetermineStrategy` decides whether to run a `PipelineSync` or a `QuickSync` strategy.

- **`PipelineSync`**: Executes the deployment using a pipeline explicitly defined by the user in `app.pipecd.yaml`.
- **`QuickSync`**: Bypasses any user-defined pipelines and executes a default set of sync stages defined by the plugin. For example, in Kubernetes deployments, a minor change (like updating replicas) might bypass canary release pipelines using a Quick Sync.

If your plugin does not contain complex logic to dynamically choose between `QuickSync` and `PipelineSync`, you can simply return `nil, nil`. In this case, Piped will fall back to the strategy configured in the application configuration.

Update `DetermineStrategy` as follows:

```go
func (plugin) DetermineStrategy(context.Context, *config, *sdk.DetermineStrategyInput[applicationConfig]) (*sdk.DetermineStrategyResponse, error) {
return nil, nil
}
```
Loading
Loading