You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: store/README.md
+42-92Lines changed: 42 additions & 92 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -165,104 +165,54 @@ These prefixes are only used internally and never exposed to the user.
165
165
166
166
## Txn: Ad Hoc Nested Transactions Implementation
167
167
168
-
The `Txn` type provides a transaction-like interface for the store, allowing for atomic operations and rollbacks. This is particularly useful for testing block proposals and managing ephemeral states.
168
+
The `Txn` type provides a transaction-like interface for the store, allowing for atomic operations and rollbacks. This is particularly useful for testing block proposals and managing ephemeral states.
169
+
One of its features is the ability to create nested transactions, enabling complex multi-level state modifications while maintaining atomicity.
169
170
170
-
### Core Components
171
+
### Nested Transactions
171
172
172
-
#### Txn Structure
173
-
The `Txn` type consists of three main structures:
173
+
Nested transactions in Canopy's `Txn` implementation provide a code-level abstraction for managing hierarchical state modifications.
174
+
Unlike database-level nested transactions, these are implemented entirely in memory and provide a way to group operations that can be independently committed or rolled back.
175
+
This creates an abstraction for managing complex state changes that need to be atomic at different levels, while maintaining the simplicity of BadgerDB's single-level transaction model.
174
176
175
-
1.**Txn**: The main transaction type that embeds both the parent store interface and internal transaction state
176
-
2.**txn**: Internal state structure containing:
177
-
- A map of pending operations (set/delete)
178
-
- A sorted list of keys for efficient iteration
179
-
- A length counter for the sorted list
180
-
3.**op**: Operation type that represents either a set or delete operation with its associated value
177
+
#### Key Features of Nested Transactions
178
+
179
+
1.**Hierarchical Structure**
180
+
Transactions form a hierarchy where child transactions inherit parent state but keep their own changes isolated until committed. Parents can roll back child changes atomically.
181
+
182
+
2.**Independent Commit/Rollback**
183
+
Each transaction level can be committed or rolled back independently. Child changes affect the parent only when committed, and parent rollbacks undo all nested changes.
184
+
185
+
3.**State Isolation**
186
+
Changes are confined to the transaction level where they occur. Parents don't see uncommitted child changes; children can access committed parent state.
187
+
188
+
4.**Atomic Operations**
189
+
All changes within a transaction level are atomic. Commits and rollbacks affect only that level and its children, preserving integrity and consistency.
190
+
191
+
### What `Txn` Does
192
+
193
+
1.**Wraps a Parent Store**: Acts as a temporary overlay on top of an existing store, enabling nested transaction hierarchies
194
+
2.**Stores Changes in Memory**: All operations (set/delete) are kept in memory until explicitly written, allowing child transactions to maintain isolated state
195
+
3.**Supports Atomic Writes**: All in-memory changes can be committed at once or discarded, maintaining atomicity at each transaction level
196
+
4.**Enables Rollbacks**: Discarding reverts all uncommitted operations, including those from child transactions
197
+
5.**Supports Iteration**: Provides forward and reverse iteration with in-memory and base-store merging, respecting transaction hierarchy
198
+
199
+
### How It Works
200
+
201
+
The `Txn` system has three key internal parts that work together to support nested transactions:
202
+
203
+
-**`Txn`**: The main transaction object exposed to users, capable of creating child transactions
204
+
-**`txn`**: A struct that holds:
205
+
- A map of pending operations, maintaining isolation between transaction levels
206
+
- A sorted list of keys for fast iteration across the transaction hierarchy
207
+
-**`op`**: Represents an individual operation (Set or Delete) that can be committed or rolled back at any transaction level
181
208
182
209
### Key Features
183
210
184
-
-**In-Memory Operations**:
185
-
- All write operations (Set/Delete) are stored in memory until explicitly written to the parent store
186
-
- Provides fast access to pending changes without disk I/O
187
-
- Enables efficient rollback of operations before they're committed
188
-
189
-
-**Atomic Operations**:
190
-
- Write() method ensures all operations are applied atomically to the parent store
191
-
- Either all operations succeed or none are applied
192
-
- Maintains data consistency even during concurrent access
193
-
194
-
-**Rollback Support**:
195
-
- Discard() method allows rolling back all pending operations
196
-
- Useful for testing block proposals or handling failed operations
197
-
- Cleans up memory resources associated with pending operations
198
-
199
-
-**Efficient Iteration**:
200
-
- Maintains sorted keys for efficient iteration and merging with parent store
201
-
- Supports both forward and reverse iteration patterns
202
-
- Enables efficient range queries and prefix-based filtering
203
-
204
-
-**Prefix-Based Iteration**:
205
-
- Supports both forward and reverse iteration with prefix filtering
206
-
- Enables efficient scanning of related data
207
-
- Useful for batch operations on related keys
208
-
209
-
### Performance Considerations
210
-
211
-
-**Memory Usage**:
212
-
- All operations are kept in memory until Write() or Discard()
213
-
- Memory footprint grows linearly with the number of pending operations
214
-
- Efficient memory management through buffer pooling and automatic cleanup
215
-
216
-
-**Iteration Efficiency**:
217
-
- Uses binary search for O(log n) key lookups in sorted list
218
-
- Maintains sorted order for efficient range queries
219
-
- Optimized merging of in-memory and parent store data during iteration
220
-
221
-
-**Write Performance**:
222
-
- Write() operation is O(n) where n is the number of pending operations
223
-
- Batch operations are more efficient than individual writes
224
-
- Memory operations are significantly faster than disk operations
225
-
226
-
-**Read Performance**:
227
-
- Get() is O(1) for in-memory operations, falls back to parent store
228
-
- In-memory operations take precedence over parent store data
229
-
- Efficient key lookup through hash map and sorted list combination
230
-
231
-
### Implementation Details
232
-
233
-
#### Write Operations
234
-
- Set() and Delete() operations are stored in an in-memory map
235
-
- Keys are automatically added to a sorted list for efficient iteration
236
-
- Operations are not applied to the parent store until Write() is called
237
-
- The sorted list enables efficient binary search for key lookups
238
-
239
-
#### Read Operations
240
-
- Get() first checks the in-memory operations, then falls back to the parent store
241
-
- Iterator() and RevIterator() merge in-memory operations with parent store data
242
-
- Deleted keys are properly handled during iteration
243
-
- Prefix-based filtering is supported for both forward and reverse iteration
244
-
245
-
#### Memory Management
246
-
- Uses a map for O(1) operation lookups
247
-
- Maintains a sorted slice for efficient iteration
248
-
- Implements buffer pooling for memory efficiency
249
-
- Automatically manages memory for pending operations
250
-
251
-
### Usage Example
252
-
253
-
The Txn type is typically used in scenarios requiring atomic operations or rollback capabilities:
254
-
255
-
1. Create a new transaction with a parent store
256
-
2. Perform multiple Set() and Delete() operations
257
-
3. Either commit the changes using Write() or rollback using Discard()
258
-
4. The transaction maintains all operations in memory until explicitly committed
259
-
260
-
### Limitations
261
-
262
-
- Not thread-safe (should not be used across multiple goroutines)
263
-
- Write() is not atomic when writing to another memory store
264
-
- Keys must be smaller than 128 bytes
265
-
- Nested transactions are supported but iteration becomes increasingly inefficient with depth
211
+
-**In-Memory Speed**: Fast reads/writes with no disk access, enabling efficient nested transaction operations
212
+
-**Efficient Iteration**: Maintains sorted keys for quick scanning and merging across transaction levels
213
+
-**Read Precedence**: Reads prioritize in-memory updates over the underlying store, respecting transaction hierarchy
214
+
-**Atomic Commit**: `Write()` commits all changes at once to the parent store, maintaining atomicity at each level
215
+
-**Safe Rollback**: `Discard()` wipes all uncommitted changes, including those from child transactions
0 commit comments