Skip to content

Comments

Start-DbaDbEncryption: Fix issues with parallel processing#10083

Merged
potatoqualitee merged 3 commits intodevelopmentfrom
fix_Start-DbaDbEncryption
Jan 11, 2026
Merged

Start-DbaDbEncryption: Fix issues with parallel processing#10083
potatoqualitee merged 3 commits intodevelopmentfrom
fix_Start-DbaDbEncryption

Conversation

@andreasjordan
Copy link
Collaborator

Type of Change

  • Bug fix (non-breaking change, fixes # )
  • New feature (non-breaking change, adds functionality, fixes # )
  • Breaking change (affects multiple commands or functionality, fixes # )
  • Ran manual Pester test and has passed (Invoke-ManualPester)
  • Adding code coverage to existing functionality
  • Pester test is included
  • If new file reference added for test, has is been added to github.com/dataplat/appveyor-lab ?
  • Unit test is included
  • Documentation
  • Build system

Best fix would be fix the import of dbatools inside of the runspaces.

This is just a workaround.

The diff is maybe difficult to read as I fixed the indentation as well.

@potatoqualitee
Copy link
Member

hell yeah, excited about improvements to the encryption suite

@potatoqualitee
Copy link
Member

on the agenda, just a bit to review

@andreasjordan
Copy link
Collaborator Author

Maybe you find a way to import dbatools inside of the runspace so that the custom methods like ".Query()" are imported correctly. Then we would have a blueprint for that kind of parallelism. That would be even better than my fix.

I forgot to take a screenshot of the error messages. But you just need to run this part of the test:

            $parallelBackupPath = "$($TestConfig.Temp)\$CommandName-Parallel-$(Get-Random)"
            $null = New-Item -Path $parallelBackupPath -ItemType Directory
            $parallelTestDatabases = @()
            1..3 | ForEach-Object {
                $parallelTestDatabases += New-DbaDatabase -SqlInstance $TestConfig.instance2
            }


            $passwd = ConvertTo-SecureString "dbatools.IO" -AsPlainText -Force
            $splatParallelEncryption = @{
                SqlInstance             = $TestConfig.instance2
                Database                = $parallelTestDatabases.Name
                MasterKeySecurePassword = $passwd
                BackupSecurePassword    = $passwd
                BackupPath              = $parallelBackupPath
                Parallel                = $true
            }
            $results = Start-DbaDbEncryption @splatParallelEncryption

@andreasjordan
Copy link
Collaborator Author

You don't even need multiple databases, should also work with just one. Only use the -Parallelswitch.

@potatoqualitee potatoqualitee merged commit 8292553 into development Jan 11, 2026
14 checks passed
@potatoqualitee potatoqualitee deleted the fix_Start-DbaDbEncryption branch January 11, 2026 18:58
@potatoqualitee
Copy link
Member

looks good, will talk to @claude about what we can do in our dbatools.dll in dbatools.library possibly to help with importing minimal parts of dbatools in parallel runspaces

@claude
Copy link

claude bot commented Jan 11, 2026

Claude finished @potatoqualitee's task in 2m 21s —— View job


Analysis: Parallel Runspace Import Challenges

I've analyzed the merged PR and the parallel processing implementation to understand the runspace module import issue.


Current State - The Workaround

The Problem:

  • When dbatools is imported into runspace pools, custom methods like .Query() aren't available
  • The workaround moves database encryption key creation outside the runspaces (Start-DbaDbEncryption.ps1:488-502)
  • This is done sequentially before parallel processing begins, because New-DbaDbEncryptionKey internally uses Get-DbaDatabase which depends on the .Query() method

Current Import Method (Start-DbaDbEncryption.ps1:558-562):

$initialSessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault()
$dbatools = Get-Module -Name dbatools
if ($dbatools) {
    $initialSessionState.ImportPSModule($dbatools.Path)
}

This approach imports the full dbatools module, but the custom methods added by dbatools.dll aren't properly initialized in the runspace.


Root Cause Analysis

The .Query() extension method comes from dbatools.library (a compiled DLL that extends SMO objects). When dbatools loads normally:

  1. dbatools.psm1 imports dbatools.library module
  2. dbatools.library loads dbatools.dll which contains extension methods
  3. These extension methods are added to SMO types in the current PowerShell session

The runspace issue: When using ImportPSModule() in a runspace's InitialSessionState, the module loads but the DLL's type extensions aren't properly registered in the new runspace's type system.


Potential Solutions

Option 1: Explicit DLL Import in Runspaces (Recommended for dbatools.library)

Add a helper method to dbatools.library that can be called to initialize extension methods in a runspace:

// In dbatools.library
public static class RunspaceInitializer 
{
    public static void InitializeExtensionMethods() 
    {
        // Re-register type extensions for current runspace
        // This would need to be called once per runspace
    }
}

Then in PowerShell:

$encryptionScript = {
    # Import just the DLL and initialize extensions
    Import-Module "$using:libraryRoot\lib\dbatools.dll"
    [Dataplat.Dbatools.RunspaceInitializer]::InitializeExtensionMethods()
    
    # Now extension methods work
    $server.Query("SELECT 1")
}

Option 2: Import DLL Directly in InitialSessionState

$initialSessionState = [InitialSessionState]::CreateDefault()
$libraryPath = Get-DbatoolsLibraryPath
$dllPath = [System.IO.Path]::Combine($libraryPath, 'lib', 'dbatools.dll')

# Import the DLL as an assembly in the session state
$initialSessionState.Assemblies.Add([System.Management.Automation.Runspaces.SessionStateAssemblyEntry]::CreateFrom($dllPath))

# Then import the module
$initialSessionState.ImportPSModule("dbatools")

Option 3: Use Variables to Pass Pre-Connected SMO Objects

Instead of connecting in runspaces, pass connected server objects:

$encryptionScript = {
    param($ServerObject, $DatabaseName)
    # Use pre-connected object that already has extensions
    $db = $ServerObject.Databases[$DatabaseName]
    # Work with it
}

Option 4: Minimal Import Pattern (Current Workaround)

Keep doing what the PR does - move any code requiring .Query() outside runspaces and only parallelize operations that don't need it.


Recommendations for dbatools.library

To properly support parallel runspaces, I recommend adding these to dbatools.library:

  1. Add a Initialize-DbaRunspace cmdlet:

    # Exported from dbatools.library
    function Initialize-DbaRunspace {
        # Ensures extension methods are available in current runspace
        # Can be called from within a runspace script block
    }
  2. Add documentation about runspace initialization requirements

  3. Create a helper for creating properly initialized InitialSessionState objects:

    function New-DbaInitialSessionState {
        # Returns an InitialSessionState with dbatools properly configured
        # Includes proper DLL assembly references
    }
  4. Export minimal required assemblies for lightweight runspace imports


Short-Term Best Practice

For now, the current workaround is solid. When using parallel runspaces with dbatools:

  1. ✅ Pre-create shared resources (master keys, certificates) sequentially
  2. ✅ Move any operations requiring .Query() outside runspaces
  3. ✅ Only parallelize operations that use standard SMO properties/methods
  4. ✅ Pass connection strings and reconnect inside runspaces

This is exactly what the merged PR does, and it's a pragmatic solution until dbatools.library can provide better runspace support.


References:

  • Start-DbaDbEncryption.ps1 (lines 488-502, 558-562)
  • dbatools.psm1 (lines 114-118 showing DLL import)
  • .github/DBATOOLS_LIBRARY_VERSION_MANAGEMENT.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants