feat: import wizard reliability and UX improvements by ManukMinasyan · Pull Request #115 · relaticle/relaticle

ManukMinasyan · 2026-02-12T14:13:57Z

Summary

Batch JSON custom field resolution — eliminates N+1 queries in EntityLinkResolver::resolveViaJsonColumn() by using chunked orWhereJsonContains clauses
Fix chunk pagination bug — chunk() with a mutating WHERE processed=false condition skipped every other chunk; switched to chunkById()
Fix null match_action crash — rows with null match_action passed through the skip guard and hit ImportDataStorage::setMultiple() with a null record; changed guard from blacklist (isSkip) to whitelist (isCreate || isUpdate)
Concurrent import prevention — Livewire state guard + Cache::lock() atomic status transition in ImportStore::transitionToImporting()
Failed rows CSV export — new downloadFailedRows Filament action on PreviewStep, streams CSV with original data + error column
UTF-8/international data tests — Japanese, Arabic, emoji, accented Latin characters
Large dataset tests — 1000-row create, mixed operations, and entity link dedup (grouped as slow)

Test plan

php artisan test --filter=ExecuteImportJob — 37 tests pass
php artisan test --filter=PreviewStep — 34 tests pass
vendor/bin/phpstan analyse — no new errors
vendor/bin/pint --dirty — clean

Introduce `CleanupImportsCommand` to remove stale and completed import files based on configurable age thresholds. Register the command and add corresponding tests to ensure behavior consistency.

- Batch JSON custom field resolution to eliminate N+1 queries - Fix chunk pagination bug that skipped rows in large imports (chunkById) - Add concurrent import prevention with Cache::lock and Livewire guards - Add failed rows CSV export/download action - Add UTF-8/international data tests and large dataset (1000-row) tests

Rows without a match_action (null) passed through the skip guard and reached ImportDataStorage::setMultiple() with a null record. Changed the guard from blacklist (isSkip) to whitelist (isCreate || isUpdate).

Copilot

Pull request overview

This PR enhances the import wizard's reliability and user experience through performance optimizations, bug fixes, and improved error handling. The changes address N+1 query issues, pagination bugs, null value crashes, and add concurrent import prevention with failed row CSV export functionality.

Changes:

Batch resolution for JSON custom field queries to eliminate N+1 performance issues during entity linking
Fixed chunk pagination bug by switching from chunk() to chunkById() to prevent record skipping
Added null match_action guard using whitelist approach instead of blacklist to prevent crashes
Implemented concurrent import prevention using cache locks and Livewire state guards
Added CSV export functionality for failed import rows with original data and error messages
Comprehensive test coverage for UTF-8/international data and large datasets (1000+ rows)

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/Feature/ImportWizard/Livewire/PreviewStepTest.php	Tests for concurrent import prevention and failed rows download action
tests/Feature/ImportWizard/Jobs/ExecuteImportJobTest.php	Tests for batch JSON queries, null match_action handling, UTF-8 data, and large datasets
tests/Feature/ImportWizard/Commands/CleanupImportsCommandTest.php	New test file for import cleanup command
app-modules/ImportWizard/src/Support/EntityLinkResolver.php	Batched JSON query resolution with chunking to eliminate N+1 queries
app-modules/ImportWizard/src/Store/ImportStore.php	Added atomic state transition with locking and failed rows tracking
app-modules/ImportWizard/src/Store/ImportRow.php	Added `processed` column for tracking row processing state
app-modules/ImportWizard/src/Livewire/Steps/PreviewStep.php	Concurrent import guards and failed rows CSV download action
app-modules/ImportWizard/src/Jobs/ExecuteImportJob.php	Fixed pagination, null guards, transaction wrapping, notifications, and error tracking
app-modules/ImportWizard/src/Commands/CleanupImportsCommand.php	New command for cleaning up stale and completed imports
app-modules/ImportWizard/src/ImportWizardNewServiceProvider.php	Registered cleanup command
app-modules/ImportWizard/resources/views/livewire/steps/preview-step.blade.php	Updated UI to show failed count and download button
routes/console.php	Removed schedule registration (moved to bootstrap/app.php)
composer.json	Updated custom-fields package constraint from @dev to ^3.0-beta
composer.lock	Updated dependencies: relaticle/custom-fields, spatie/laravel-medialibrary, laravel/boost, laravel/roster

Comments suppressed due to low confidence (1)

app-modules/ImportWizard/src/Support/EntityLinkResolver.php:145

The removal of withoutGlobalScopes() on this line may cause issues if the target model class has tenant scoping or other global scopes. The query already filters by team_id, but if the model has a global scope that applies additional constraints (e.g., soft deletes, multi-tenancy), removing this call could prevent matching existing records during import.

        return $modelClass::query()
            ->where('team_id', $this->teamId)
            ->whereIn($field, $uniqueValues)
            ->pluck('id', $field)
            ->all();

Copilot · 2026-02-12T14:20:35Z

app-modules/ImportWizard/src/Jobs/ExecuteImportJob.php

@@ -181,7 +242,6 @@ private function findExistingRecord(BaseImporter $importer, ?string $matchedId):
        $modelClass = $importer->modelClass();

        return $modelClass::query()


The removal of withoutGlobalScopes() on this line may cause issues if the model has global scopes (e.g., soft deletes, tenant scoping). Since this is used to find existing records for updates during import, failing to bypass global scopes could prevent legitimate matches from being found.

Suggested change

return $modelClass::query()

return $modelClass::withoutGlobalScopes()

Copilot · 2026-02-12T14:20:36Z

composer.json

        "livewire/livewire": "^4.0",
        "prism-php/prism": "^0.98.5",
-        "relaticle/custom-fields": "@dev",
+        "relaticle/custom-fields": "  ^3.0-beta",


There are extra spaces before the caret in the version constraint. This should be ^3.0-beta instead of ^3.0-beta to maintain consistent formatting with the rest of the file.

Suggested change

"relaticle/custom-fields": " ^3.0-beta",

"relaticle/custom-fields": "^3.0-beta",

Copilot · 2026-02-12T14:20:36Z

app-modules/ImportWizard/resources/views/livewire/steps/preview-step.blade.php

                    <div class="flex items-center gap-4 text-xs ml-auto shrink-0">
                        <span class="flex items-center gap-1.5">
                            <span class="font-semibold text-success-700 dark:text-success-300">{{ number_format($this->results['created'] ?? 0) }}</span>
                            <span class="text-gray-500 dark:text-gray-400">created</span>
                        </span>
                        <span class="flex items-center gap-1.5">
                            <span class="font-semibold text-primary-700 dark:text-primary-300">{{ number_format($this->results['updated'] ?? 0) }}</span>
                            <span class="text-gray-500 dark:text-gray-400">updated</span>
                        </span>
                        <span class="flex items-center gap-1.5">
-                            <span class="font-semibold text-gray-600 dark:text-gray-300">{{ number_format($this->results['skipped'] ?? 0) }}</span>
-                            <span class="text-gray-500 dark:text-gray-400">skipped</span>
+                            <span class="font-semibold text-danger-700 dark:text-danger-300">{{ number_format($this->results['failed'] ?? 0) }}</span>
+                            <span class="text-gray-500 dark:text-gray-400">failed</span>
                        </span>
                    </div>


The UI now displays only "created", "updated", and "failed" counts, but the results array still tracks "skipped" rows (see line 73 of ExecuteImportJob.php). While skipped rows are counted internally, users have no visibility into how many rows were skipped. Consider either displaying the skipped count in the UI or documenting why it's intentionally hidden.

Copilot · 2026-02-12T14:20:36Z

app-modules/ImportWizard/src/Jobs/ExecuteImportJob.php

+            });
+
+            $this->markProcessed($row);


The markProcessed() call on line 162 occurs outside the database transaction (lines 138-160). If the transaction succeeds but markProcessed() fails, the row will be successfully imported but not marked as processed. On job retry (configured with 3 attempts and backoff), this could lead to duplicate records being created. Consider moving markProcessed() inside the transaction or using a different strategy to ensure atomicity.

Suggested change

});

$this->markProcessed($row);

$this->markProcessed($row);

});

ManukMinasyan added 3 commits February 12, 2026 18:13

feat: add CleanupImportsCommand for stale file removal

8a0d5f4

Introduce `CleanupImportsCommand` to remove stale and completed import files based on configurable age thresholds. Register the command and add corresponding tests to ensure behavior consistency.

fix: skip rows with null match_action to prevent null record crash

2a75a22

Rows without a match_action (null) passed through the skip guard and reached ImportDataStorage::setMultiple() with a null record. Changed the guard from blacklist (isSkip) to whitelist (isCreate || isUpdate).

Copilot AI review requested due to automatic review settings February 12, 2026 14:13

ManukMinasyan merged commit 8b6779c into 3.x Feb 12, 2026
7 of 10 checks passed

Copilot started reviewing on behalf of ManukMinasyan February 12, 2026 14:14 View session

Copilot AI reviewed Feb 12, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: import wizard reliability and UX improvements#115

feat: import wizard reliability and UX improvements#115
ManukMinasyan merged 3 commits into3.xfrom
feat/import-wizard-reliability-improvements

ManukMinasyan commented Feb 12, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 12, 2026

Uh oh!

Copilot AI Feb 12, 2026

Uh oh!

Copilot AI Feb 12, 2026

Uh oh!

Copilot AI Feb 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		@@ -181,7 +242,6 @@ private function findExistingRecord(BaseImporter $importer, ?string $matchedId):
		$modelClass = $importer->modelClass();

		return $modelClass::query()

	return $modelClass::query()
	return $modelClass::withoutGlobalScopes()

	"relaticle/custom-fields": " ^3.0-beta",
	"relaticle/custom-fields": "^3.0-beta",

Conversation

ManukMinasyan commented Feb 12, 2026

Summary

Test plan

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants