Merge pull request #21 from bdurand/delete-missing-option

Add delete_missing option to sync commands

Author: bdurand

CHANGELOG.md

@@ -4,6 +4,12 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 1.6.0
+
+### Added
+
+- Added `delete_missing` option to `sync_table_data!` and `sync_all!`. When set to `true`, any records in the database that are not defined in the data files will be deleted. This option defaults to `false` to preserve backward compatibility.
+
 ## 1.5.2
 
 ### Added

README.md

@@ -240,6 +240,21 @@ Status.sync_table_data!
 
 This will add any missing records to the table and update existing records so that the attributes in the table match the values in the data files. Records that do not appear in the data files will not be touched. Any attributes not specified in the data files will not be changed.
 
+If you want to remove records from the database that are no longer in the data files, you can pass `delete_missing: true`:
+
+```ruby
+Status.sync_table_data!(delete_missing: true)
+```
+
+This option can also be passed to `SupportTableData.sync_all!`:
+
+```ruby
+SupportTableData.sync_all!(delete_missing: true)
+```
+
+> [!CAUTION]
+> Use `delete_missing` with care. It will delete any records in the table that are not defined in the data files, which may include user-created data or fail due to foreign key constraints.
+
 The number of records contained in data files should be fairly small (ideally fewer than 100). It is possible to load just a subset of rows in a large table because only the rows listed in the data files will be synced. You can use this feature if your table allows user-entered data, but has a few rows that must exist for the code to work.
 
 Loading data is done inside a database transaction. No changes will be persisted to the database unless all rows for a model can be synced.
@@ -309,6 +324,20 @@ end
 
 You must also call `SupportTableData.sync_all!` before running your test suite. This method should be called in the test suite setup code after any data in the test database has been purged and before any tests are run.
 
+> [!TIP]
+> If you are using a truncation database cleaning strategy exclude the support tables from the tables that get truncated. Syncing data is much faster when the data is already in the database. You should use the `delete_missing` option to remove any records that are not in the data files instead of deleting all records before each test.
+
+```ruby
+# Excluding support tables from truncation in DatabaseCleaner configuration.
+RSpec.configure do |config|
+  config.before(:suite) do
+    support_tables = SupportTableData.support_table_classes.map(&:table_name)
+    DatabaseCleaner.clean_with(:truncation, except: support_tables)
+    SupportTableData.sync_all!(delete_missing: true)
+  end
+end
+```
+
 ## Installation
 
 Add this line to your application's Gemfile:

VERSION

@@ -1 +1 @@
-1.5.2
+1.6.0

lib/support_table_data.rb

@@ -49,11 +49,13 @@ def support_table_key_attribute
     end
 
     # Synchronize the rows in the table with the values defined in the data files added with
-    # `add_support_table_data`. Note that rows will not be deleted if they are no longer in
-    # the data files.
+    # `add_support_table_data`. By default, rows that are no longer present in the data files
+    # will not be deleted unless `delete_missing` is enabled.
     #
+    # @param delete_missing [Boolean] If true, then any records in the database that are not in the data
+    #   files will be deleted. Use with caution.
     # @return [Array<Hash>] List of saved changes for each record that was created or modified.
-    def sync_table_data!
+    def sync_table_data!(delete_missing: false)
       return unless table_exists?
 
       canonical_data = support_table_data.each_with_object({}) do |attributes, hash|
@@ -64,6 +66,8 @@ def sync_table_data!
 
       begin
         ActiveSupport::Notifications.instrument("support_table_data.sync", class: self) do
+          synced_ids = []
+
           transaction do
             records.each do |record|
               key = record[support_table_key_attribute].to_s
@@ -75,6 +79,8 @@ def sync_table_data!
                 changes << record.changes
                 record.save!
               end
+
+              synced_ids << record.id if attributes
             end
 
             canonical_data.each_value do |attributes|
@@ -86,6 +92,11 @@ def sync_table_data!
               end
               changes << record.changes
               record.save!
+              synced_ids << record.id
+            end
+
+            if delete_missing
+              where.not(primary_key => synced_ids).destroy_all
             end
           end
         end
@@ -382,11 +393,13 @@ def data_directory=(value)
     # when the test suite is initializing.
     #
     # @param extra_classes [Class] List of classes to force into the detected list of classes to sync.
+    # @param delete_missing [Boolean] If true, then any records in the database that are not in the data
+    #   files will be deleted from each table. Use with caution.
     # @return [Hash<Class, Array<Hash>] Hash of classes synced with a list of saved changes.
-    def sync_all!(*extra_classes)
+    def sync_all!(*extra_classes, delete_missing: false)
       changes = {}
       support_table_classes(*extra_classes).each do |klass|
-        changes[klass] = klass.sync_table_data!
+        changes[klass] = klass.sync_table_data!(delete_missing: delete_missing)
       end
       changes
     end

spec/support_table_data_spec.rb

@@ -63,6 +63,31 @@
       ])
       expect { Color.sync_table_data! }.to raise_error(SupportTableData::ValidationError, /Validation failed for Color with id: 20 - Name can't be blank/)
     end
+
+    it "does not delete extra rows by default" do
+      Group.sync_table_data!
+      extra = Group.new(name: "extra")
+      extra.group_id = 99
+      extra.save!
+      Group.sync_table_data!
+      expect(Group.find_by(group_id: 99)).to eq extra
+    end
+
+    it "deletes extra rows not in the data files when delete_missing is true" do
+      Group.sync_table_data!
+      extra = Group.new(name: "extra")
+      extra.group_id = 99
+      extra.save!
+      expect(Group.find_by(group_id: 99)).to eq extra
+      Group.sync_table_data!(delete_missing: true)
+      expect(Group.find_by(group_id: 99)).to be_nil
+    end
+
+    it "does not delete rows that are in the data files when delete_missing is true" do
+      Group.sync_table_data!(delete_missing: true)
+      expect(Group.count).to eq 3
+      expect(Group.pluck(:name)).to match_array ["primary", "secondary", "gray"]
+    end
   end
 
   describe "sync_all!" do
@@ -104,6 +129,16 @@
       expect(Color.find_by(id: 10)).to eq color
     end
 
+    it "deletes extra rows not in the data files when delete_missing is true" do
+      SupportTableData.sync_all!
+      extra = Group.new(name: "extra")
+      extra.group_id = 99
+      extra.save!
+      expect(Group.find_by(group_id: 99)).to eq extra
+      SupportTableData.sync_all!(delete_missing: true)
+      expect(Group.find_by(group_id: 99)).to be_nil
+    end
+
     it "combines data when a record is defined across multiple data files" do
       SupportTableData.sync_all!
       expect(purple.name).to eq "Purple"