Skip to content

Commit 71d2292

Browse files
dcharles525dcharles525
authored
Merge pull request #449 from epinio/storage-recommendation
Storage recommendation documentation added
2 parents 1953ce0 + 278d703 commit 71d2292

3 files changed

Lines changed: 320 additions & 1 deletion

File tree

‎docs/installation/install_epinio.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,10 @@ You can use any storage provisioner which provides `ReadWriteMany` (RWX) Access
104104

105105
To verify that your cluster provides a default StorageClass run the command `kubectl get storageclass`. The default StorageClass is marked with the string `(default)` next to its name in the output list.
106106

107+
:::tip Storage sizing
108+
Ensure your storage provisioner has adequate capacity for your expected workload. Storage requirements scale with the number of applications and their sizes. See [Storage recommendations](../references/system_requirements/storage.md) for detailed guidance on calculating storage needs.
109+
:::
110+
107111
For example, you can deploy and configure the `local-path` dynamic storage provisioner by:
108112
```bash
109113
kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml

‎docs/references/system_requirements/global.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ It's almost certain that you'll need more resources for typical development and
1818
| OS/Kubernetes <sup>1</sup> | Linux: [RKE2](../../installation/other_inst_scenarios/install_epinio_on_rke.md), [K3s](../../installation/other_inst_scenarios/install_epinio_on_k3s.md), [K3d](../../installation/other_inst_scenarios/install_epinio_on_k3d.md), [Rancher Desktop](../../installation/other_inst_scenarios/install_epinio_on_rancher_desktop.md)<br/>Windows, macOS: [Rancher Desktop](../../installation/other_inst_scenarios/install_epinio_on_rancher_desktop.md) |
1919
| CPU | 2–4 vCPUs |
2020
| Memory | 8 GB RAM (system memory + 4 GB) |
21-
| Storage | 10 GB Disk space (system disk + 5 GB) |
21+
| Storage | 10 GB Disk space minimum (system disk + 5 GB)<br/>**See [Storage recommendations](storage.md) for production deployments** |
2222

2323
<sup>1</sup> Linux: x86_64, arm64 and s390x; macOS: x86_64, arm64 (Epinio CLI only); Windows: x86_64
2424

‎docs/references/system_requirements/storage.md‎

Lines changed: 315 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,315 @@
1+
---
2+
sidebar_label: Storage recommendations
3+
title: Storage recommendations
4+
description: Storage recommendations for Epinio based on scale of users and applications.
5+
keywords: [epinio, kubernetes, storage, disk space, scale, requirements]
6+
doc-type: [reference]
7+
doc-topic: [epinio/reference/storage]
8+
doc-persona: [epinio-operator, epinio-developer]
9+
---
10+
11+
## Overview
12+
13+
Storage configuration and requirements are critical for maintaining Epinio. Users that underprovision storage space could be blocked fairly quickly as users continue to onboard to a particular instance. This document provides guidance on storage requirements and recommendations based on the scale of your Epinio deployment.
14+
15+
## Storage Components
16+
17+
Epinio uses several storage components that need to be considered when planning your deployment:
18+
19+
### 1. Application Source Blobs (S3 Storage)
20+
21+
**Purpose**: Stores uploaded application source code (tarballs, archives) before staging.
22+
23+
**Storage Type**: S3-compatible object storage (internal MinIO or external S3)
24+
25+
**Calculation**:
26+
- Each application upload creates one blob in S3
27+
- Storage = `Number of apps × Average source blob size × Retention factor`
28+
- Typical source blob sizes: 10-100 MB (small apps), 100-500 MB (medium apps), 500 MB-2 GB (large apps)
29+
- Retention factor: Consider how many versions/upload attempts you want to retain (default: all)
30+
31+
**Example**:
32+
- 50 apps with average 50 MB source blobs = ~2.5 GB minimum
33+
- With 3 versions per app retained = ~7.5 GB
34+
35+
**Configuration Impact**:
36+
- **Internal S3 (MinIO)**: Storage is provisioned within the cluster via PVC
37+
- **External S3**: Storage is managed externally (AWS S3, MinIO, etc.) - no cluster storage needed, but requires external capacity planning
38+
39+
### 2. Application Build Cache (PVC)
40+
41+
**Purpose**: Stores build cache to speed up subsequent builds of the same application.
42+
43+
**Storage Type**: Kubernetes PersistentVolumeClaim (PVC) or EmptyDir
44+
45+
**Default Size**: 1 GiB per application (if not specified)
46+
47+
**Calculation**:
48+
- If using PVCs: `Number of apps × Cache size per app`
49+
- Default: `N apps × 1 GiB = N GiB`
50+
- Cache size depends on buildpack layers and application dependencies
51+
- Typical cache sizes: 500 MB - 2 GB per app
52+
53+
**Configuration Options**:
54+
- **Persistent (PVC)**: Cache persists across builds, faster rebuilds, requires storage
55+
- **EmptyDir**: Ephemeral, no storage provisioning needed, but slower rebuilds
56+
57+
**Example**:
58+
- 50 apps with 1 GiB cache each = 50 GiB total
59+
60+
### 3. Application Source Blobs (PVC)
61+
62+
**Purpose**: Stores application source code during staging process.
63+
64+
**Storage Type**: Kubernetes PersistentVolumeClaim (PVC) or EmptyDir
65+
66+
**Default Size**: 1 GiB per application (if not specified)
67+
68+
**Calculation**:
69+
- If using PVCs: `Number of apps × Source blob size per app`
70+
- Default: `N apps × 1 GiB = N GiB`
71+
- Should accommodate largest application source archive
72+
73+
**Configuration Options**:
74+
- **Persistent (PVC)**: Source persists, allows reuse, requires storage
75+
- **EmptyDir**: Ephemeral, downloaded fresh each time, no storage provisioning needed
76+
77+
**Example**:
78+
- 50 apps with 1 GiB source storage each = 50 GiB total
79+
80+
### 4. Container Registry
81+
82+
**Purpose**: Stores built application container images.
83+
84+
**Storage Type**: Container registry storage (internal or external)
85+
86+
**Calculation**:
87+
- Storage = `Number of apps × Number of versions × Average image size`
88+
- Image size = Base image layers + Application layers + Buildpack layers
89+
- Typical image sizes: 100-300 MB (small apps), 300-800 MB (medium apps), 800 MB-2 GB (large apps)
90+
- Each new stage creates a new image version
91+
92+
**Example**:
93+
- 50 apps with average 500 MB images, 3 versions each = ~75 GB
94+
95+
**Configuration Impact**:
96+
- **Internal Registry**: Storage is provisioned within the cluster
97+
- **External Registry**: Storage is managed externally (Docker Hub, AWS ECR, etc.) - no cluster storage needed
98+
99+
## Storage Calculation Formulas
100+
101+
### Total Cluster Storage (Internal Components Only)
102+
103+
If using internal S3 and internal registry:
104+
105+
```
106+
Total Storage = S3 Storage + (Cache PVCs + Source Blobs PVCs) + Registry Storage
107+
```
108+
109+
Where:
110+
- **S3 Storage** = `N apps × Avg source blob size × Retention factor`
111+
- **Cache PVCs** = `N apps × Cache size` (if using PVCs, else 0)
112+
- **Source Blobs PVCs** = `N apps × Source blob size` (if using PVCs, else 0)
113+
- **Registry Storage** = `N apps × Avg image size × Versions per app`
114+
115+
### Simplified Formula
116+
117+
For quick estimation with default configurations:
118+
119+
```
120+
Total Storage (GB) ≈ N apps × (Source blob size + Cache size + Source PVC size + Image size × Versions)
121+
```
122+
123+
With typical defaults (1 GiB cache, 1 GiB source PVC, 50 MB source blob, 500 MB image, 3 versions):
124+
```
125+
Total Storage (GB) ≈ N apps × (0.05 + 1 + 1 + 0.5 × 3) = N apps × 3.55 GB
126+
```
127+
128+
## Recommendations by Scale
129+
130+
### Small Scale (1-10 apps)
131+
132+
**Storage Requirements**:
133+
- **S3 Storage**: 1-5 GB (depending on source blob sizes)
134+
- **PVC Storage** (if using persistent): 20-40 GB (2-4 GB per app)
135+
- **Registry Storage**: 5-30 GB (depending on image sizes and versions)
136+
- **Total**: ~30-75 GB
137+
138+
**Recommendations**:
139+
- Default 1 GiB PVC sizes are typically sufficient
140+
- Can use EmptyDir for cache/source if storage is constrained
141+
- Internal S3 and registry are fine for this scale
142+
143+
### Medium Scale (10-50 apps)
144+
145+
**Storage Requirements**:
146+
- **S3 Storage**: 5-25 GB
147+
- **PVC Storage** (if using persistent): 100-200 GB
148+
- **Registry Storage**: 30-150 GB
149+
- **Total**: ~150-400 GB
150+
151+
**Recommendations**:
152+
- Monitor PVC usage and adjust sizes based on actual application sizes
153+
- Consider external S3 if cluster storage is limited
154+
- Plan for registry storage growth as apps are updated
155+
- Consider using storage classes with dynamic provisioning
156+
157+
### Large Scale (50-200 apps)
158+
159+
**Storage Requirements**:
160+
- **S3 Storage**: 25-100 GB
161+
- **PVC Storage** (if using persistent): 200-800 GB
162+
- **Registry Storage**: 150-600 GB
163+
- **Total**: ~400 GB - 1.5 TB
164+
165+
**Recommendations**:
166+
- **Strongly consider external S3** to reduce cluster storage pressure
167+
- Use external container registry for better scalability
168+
- Implement storage quotas and monitoring
169+
- Use appropriate storage classes (e.g., SSD for cache, standard for source)
170+
- Plan for regular cleanup of old images and source blobs
171+
172+
### Enterprise Scale (200+ apps)
173+
174+
**Storage Requirements**:
175+
- **S3 Storage**: 100+ GB
176+
- **PVC Storage** (if using persistent): 800+ GB
177+
- **Registry Storage**: 600+ GB
178+
- **Total**: 1.5+ TB
179+
180+
**Recommendations**:
181+
- **Use external S3** (AWS S3, MinIO, etc.)
182+
- **Use external container registry** (AWS ECR, Harbor, etc.)
183+
- Implement automated cleanup policies
184+
- Use storage classes optimized for workload (high IOPS for cache)
185+
- Monitor and alert on storage usage
186+
- Consider storage tiering strategies
187+
188+
## Configuration Options and Their Impact
189+
190+
### Storage Class Configuration
191+
192+
**Impact**: Different storage classes have different performance characteristics and costs.
193+
194+
- **Standard/HDD**: Lower cost, suitable for source blobs and S3
195+
- **SSD/Fast**: Higher cost, better for cache volumes that benefit from IOPS
196+
- **External Storage Classes**: May have different provisioning models (e.g., cloud storage with different tiers)
197+
198+
**Recommendation**: Use faster storage classes for cache volumes if budget allows, as they significantly improve build times.
199+
200+
### EmptyDir vs Persistent Volumes
201+
202+
**EmptyDir (Ephemeral)**:
203+
- **Pros**: No storage provisioning needed, no PVC management
204+
- **Cons**: Slower builds (no cache reuse), data lost on pod restart
205+
- **Use Case**: Development environments, storage-constrained deployments
206+
207+
**Persistent Volumes (PVC)**:
208+
- **Pros**: Faster builds (cache reuse), data persistence
209+
- **Cons**: Requires storage provisioning, PVC management overhead
210+
- **Use Case**: Production environments, frequent rebuilds
211+
212+
### External S3 Configuration
213+
214+
**Impact**: Moves source blob storage outside the cluster.
215+
216+
- **Pros**: Reduces cluster storage requirements, better scalability, can leverage cloud storage features
217+
- **Cons**: Requires external S3 setup and credentials, network dependency
218+
- **Storage Impact**: Eliminates S3 storage from cluster storage calculations
219+
220+
**When to Use**:
221+
- Medium to large scale deployments
222+
- Limited cluster storage capacity
223+
- Need for S3-specific features (lifecycle policies, cross-region replication, etc.)
224+
225+
### External Container Registry
226+
227+
**Impact**: Moves container image storage outside the cluster.
228+
229+
- **Pros**: Reduces cluster storage requirements, better scalability, can leverage registry features
230+
- **Cons**: Requires external registry setup, network dependency
231+
- **Storage Impact**: Eliminates registry storage from cluster storage calculations
232+
233+
**When to Use**:
234+
- Medium to large scale deployments
235+
- Limited cluster storage capacity
236+
- Need for registry-specific features (vulnerability scanning, image signing, etc.)
237+
238+
## Application Size Considerations
239+
240+
**Critical Note**: Storage requirements are highly dependent on application size. The formulas above use averages, but actual requirements can vary significantly.
241+
242+
**Factors Affecting Storage**:
243+
- **Source Code Size**: Larger applications require more S3 and source PVC storage
244+
- **Dependencies**: Applications with many dependencies create larger build caches and images
245+
- **Buildpack Layers**: Different buildpacks add different amounts to image size
246+
- **Multi-stage Builds**: Applications with multiple build stages may create larger intermediate artifacts
247+
248+
**Recommendation**: Monitor actual usage and adjust storage sizes based on your specific application profiles. Start with defaults and scale based on observed usage patterns.
249+
250+
## Monitoring and Maintenance
251+
252+
### Key Metrics to Monitor
253+
254+
1. **PVC Usage**: Monitor usage of cache and source blob PVCs
255+
2. **S3 Bucket Size**: Track S3 bucket growth over time
256+
3. **Registry Storage**: Monitor container registry storage usage
257+
4. **Storage Class Performance**: Track IOPS and latency for cache volumes
258+
259+
### Cleanup Strategies
260+
261+
1. **Old Source Blobs**: Implement lifecycle policies to delete old S3 objects
262+
2. **Old Images**: Configure registry garbage collection for unused images
263+
3. **Unused PVCs**: Clean up PVCs for deleted applications
264+
4. **Build Cache**: Periodically clear build caches for applications that haven't been rebuilt recently
265+
266+
## Example Scenarios
267+
268+
### Scenario 1: Small Development Environment
269+
- **10 apps**, average 30 MB source, 200 MB images, 2 versions
270+
- **Configuration**: Internal S3, internal registry, EmptyDir for cache/source
271+
- **Storage Needed**: ~3 GB (S3) + ~4 GB (registry) = **~7 GB total**
272+
273+
### Scenario 2: Medium Production Environment
274+
- **30 apps**, average 100 MB source, 500 MB images, 5 versions
275+
- **Configuration**: Internal S3, internal registry, PVCs (1 GiB each)
276+
- **Storage Needed**: ~3 GB (S3) + ~60 GB (PVCs) + ~75 GB (registry) = **~140 GB total**
277+
278+
### Scenario 3: Large Production Environment
279+
- **100 apps**, average 150 MB source, 800 MB images, 10 versions
280+
- **Configuration**: External S3, external registry, PVCs (2 GiB cache, 1 GiB source)
281+
- **Storage Needed**: ~300 GB (PVCs) + External S3/Registry = **~300 GB cluster storage** (S3/registry external)
282+
283+
## Summary
284+
285+
Storage requirements for Epinio scale with:
286+
- Number of applications
287+
- Application and image sizes
288+
- Number of versions retained
289+
- Storage configuration choices (PVC vs EmptyDir, internal vs external)
290+
291+
**Key Takeaway**: Our storage hinges largely on application size, so ensure you make note of this in your planning. Monitor actual usage and adjust accordingly. For larger deployments, consider external S3 and registry to reduce cluster storage pressure.
292+
293+
## Further Reading
294+
295+
### Epinio-Specific Resources
296+
297+
- **[Epinio Meets s3gw](https://www.suse.com/c/rancher_blog/epino-meets-s3gw/)** - Blog post discussing integrating Epinio with s3gw, a lightweight S3-compatible service, highlighting storage configurations and considerations.
298+
299+
- **[Customizing and Securing Your Epinio Installation](https://www.suse.com/c/rancher_blog/customizing-and-securing-your-epinio-installation/)** - Article exploring various customization options for Epinio, including asset storage configurations and guidance on integrating external object storage and registries.
300+
301+
- **[How to Setup External S3 Storage](https://docs.epinio.io/1.6.1/howtos/setup_external_s3)** - Epinio documentation guide on configuring external S3-compatible storage for more flexible and scalable storage solutions.
302+
303+
### General Storage Planning and Best Practices
304+
305+
- **[Eight Things to Consider Before Moving Your Storage Backup to the Cloud](https://www.eweek.com/storage/eight-things-to-consider-before-moving-your-storage-backup-to-the-cloud/)** - Key factors to evaluate when transitioning storage backups to cloud environments, including data security and mobility considerations.
306+
307+
- **[Data Resilience: Eon's Approach & Google Cloud Best Practices](https://cloud.google.com/blog/products/storage-data-transfer/data-resilience-eons-approach--google-cloud-best-practices)** - Best practices for data protection in cloud environments, emphasizing versioning, retention policies, and designing for granular recovery.
308+
309+
- **[Top 10 Tips and Tricks for Storage in the Cloud](https://bluexp.netapp.com/hubfs/Paid-Tips-and-Tricks-to-Storage-in-the-Cloud-eBook.pdf)** - eBook offering practical advice for optimizing cloud storage, covering topics like cost management, data security, and performance optimization.
310+
311+
### Kubernetes Storage Resources
312+
313+
- **[Kubernetes Storage Documentation](https://kubernetes.io/docs/concepts/storage/)** - Official Kubernetes documentation on storage concepts, including PersistentVolumes, StorageClasses, and volume types.
314+
315+
- **[Kubernetes Storage Best Practices](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#best-practices)** - Kubernetes best practices for managing persistent storage in production environments.

0 commit comments

Comments
 (0)