Automating SSISDB Migration 4/4 – Copying Object Parameter Changes with Powershell

Last in this four-part series on automating a non-wizard SSIS migration (Part one, two and three).

To recap, previous scripts covered exporting both MSDB and SSISDB projects and then – after manual upgrade/deployment- copying environments and their references. The final piece of the puzzle is to copy object parameter values that have been manually overriden – where a value is shown in bold in SSMS after clicking Configure on a project/package and the ‘Edit Value’ has some manually-set value;

I wrote a function Copy-SSISObjectParameters to do this. It runs at Project level, so takes a Project name along with source/target servers and Folder name as parameters. As in the last post’s script for copying environments, I’m using the inbuilt SSISDB procedure [catalog].[set_object_parameter_value] to wrap the process.

Because of the way Powershell handles passing boolean values, I had to do some plumbing involving creating and passing a BIT variable in the SQL batch (the proc will accept a numeric 1 or 0 but will essentially corrupt the parameter making it uneditable in the GUI), otherwise it’s quite straightforward, and works for both Project and Package level ‘overrides’. The -WhatIf switch outputs the SQL statements it would normally run without the switch.

Example – Copy-SSISObjectParameters -SourceServer SQLSSIS2014 -TargetServer SQLSSIS2017 -FolderName DailyETL -ProjectName LoadDailySales

Unfortunately the Code Syntax Block plugin isn’t displaying the full code nicely below and it would be an awful lot of manual editing to fix, especially inappropriate for a blog post series on automation, but it’s working as normal on github – Copy-SSISObjectParameters

And that’s probably it for my Powershell-SSIS repo for now, pending further migration work that might need something adding, and pending the fate of SSIS in the face of ADF, for that matter!

function Copy-SSISObjectParameters
 {
 param
 (
     [Parameter(position=0, mandatory=$true)][string]$SourceServer,
     [Parameter(position=1, mandatory=$true)][string]$TargetServer,
     [Parameter(position=2, mandatory=$true)][string]$FolderName,
     [Parameter(position=3, mandatory=$true)][string]$ProjectName,
     [Parameter(position=4, mandatory=$false)][switch]$WhatIf
 )
 <#
 .SYNOPSIS
 Copies object parameter overrides from a SSIS project on a source instance to a target instance.
 .DESCRIPTION
 Copies object parameter overrides - that is, 'Edited' values on Configuration dialog - from a SSIS project 
 on a source instance to a target instance. This includes both project and package-level overrides,
 but does not include environment references, which are copied as part of the Copy-SSISEnvironments function.
 .PARAMETER SourceServer
 Specifies the source SQL instance name
 .PARAMETER TargetServer
 Specifies the target SQL instance name
 .PARAMETER FolderName
 Specifies the SSISDB folder the project is in. Case sensitive.
 .PARAMETER ProjectName
 Specifies the SSIS project (including packages) to copy the object parameter overrides from and to. Case sensitive.
 .PARAMETER WhatIf
 Outputs generated SQL statements only - does not execute them or make any other changes.
 .OUTPUTS
 No direct outputs from function - updates target server SSISDB object parameters
 .EXAMPLE
 PS> Copy-SSISObjectParameters -SourceServer SQLSSIS2014 -TargetServer SQLSSIS2017 -FolderName DailyETL -ProjectName LoadDailySales
 Sets project/package parameter overrides in SSIS project DailyETL/LoadDailySales on server SQLSSIS2017 to the same as the project on SQLSSIS2014
 >
 Import-Module SqlServer
 Check that folders and projects exist on both servers
 $checkexistsquery = "SELECT 1 FROM [catalog].[folders] f
                             INNER JOIN [catalog].[projects] p
                                 ON f.folder_id = p.folder_id
                             WHERE f.[name] = '$($FolderName)'
                                 AND p.[name] = '$($ProjectName)'"
 try
 {
     $check = Invoke-Sqlcmd -ServerInstance $SourceServer -Database SSISDB -Query $checkexistsquery -ConnectionTimeout 5 -QueryTimeout 5 -ErrorAction Stop
 }
 catch
 {
     Write-Output "Error connecting to source server."
     Write-Output $error[0]
     return
 }
 if(!$check)
 {
     Write-Output "Error: Folder or project doesn't exist on source server."
     return
 }
 try
 {
     $check = Invoke-Sqlcmd -ServerInstance $TargetServer -Database SSISDB -Query $checkexistsquery -ConnectionTimeout 5 -QueryTimeout 5 -ErrorAction Stop
 }
 catch
 {
     Write-Output "Error connecting to target server."
     Write-Output $error[0]
     return
 }
 if(!$check)
 {
     Write-Output "Error: Folder or project doesn't exist on target server."
     return
 }
 Get manually overridden parameter values (value_type V)
 $getoverridevaluesquery = "SELECT op.object_type, op.object_name, op.parameter_name, op.data_type, op.default_value
                             FROM [catalog].[object_parameters] op
                             INNER JOIN [catalog].[projects] p
                                 ON op.project_id = p.project_id
                             INNER JOIN [catalog].[folders] f
                                 ON p.folder_id = f.folder_id
                             WHERE f.[name] = '$($FolderName)'
                             AND p.[name] = '$($ProjectName)'
                             AND op.value_type = 'V'
                             AND op.value_set = 1"
 $overridevalues = Invoke-Sqlcmd -ServerInstance $SourceServer -Database SSISDB -Query $getoverridevaluesquery -ConnectionTimeout 3 -QueryTimeout 5 -ErrorAction Stop
 foreach($overridevalue in $overridevalues)
 {
     if($overridevalue.data_type -eq 'String')
     {
         if($overridevalue.object_type -eq 20) # Project parameter
         {
             $updateparameterquery = "EXEC [catalog].[set_object_parameter_value]
                                                                         @object_type = $($overridevalue.object_type),
                                                                         @folder_name = '$($FolderName)',
                                                                         @project_name = '$($ProjectName)',
                                                                         @parameter_name = '$($overridevalue.parameter_name)',
                                                                         @parameter_value = N'$($overridevalue.default_value)',
                                                                         @value_type = 'V'"
         }
         elseif($overridevalue.object_type -eq 30) # Package parameter
         {
             $updateparameterquery = "EXEC [catalog].[set_object_parameter_value]
                                                                         @object_type = $($overridevalue.object_type),
                                                                         @folder_name = '$($FolderName)',
                                                                         @project_name = '$($ProjectName)',
                                                                         @parameter_name = '$($overridevalue.parameter_name)',
                                                                         @parameter_value = N'$($overridevalue.default_value)',
                                                                         @object_name = '$($overridevalue.object_name)',
                                                                         @value_type = 'V'"
         }
     }
     elseif($overridevalue.data_type -eq 'Boolean')
     {
         if($overridevalue.default_value -eq 'True')
         {
             $bool_value = 1
         }
         else
         {
             $bool_value = 0
         }
     if($overridevalue.object_type -eq 20) # Project parameter     {         $updateparameterquery = "DECLARE @var BIT;                                  SET @var = $($bool_value);                                  EXEC [catalog].[set_object_parameter_value]                                                                     @object_type = $($overridevalue.object_type),                                                                     @folder_name = '$($FolderName)',                                                                     @project_name = '$($ProjectName)',                                                                     @parameter_name = '$($overridevalue.parameter_name)',                                                                     @parameter_value = @var,                                                                     @value_type = 'V'"     }     elseif($overridevalue.object_type -eq 30) # Package parameter     {         $updateparameterquery = "DECLARE @var BIT;                                  SET @var = $($bool_value);                                  EXEC [catalog].[set_object_parameter_value]                                                                     @object_type = $($overridevalue.object_type),                                                                     @folder_name = '$($FolderName)',                                                                     @project_name = '$($ProjectName)',                                                                     @parameter_name = '$($overridevalue.parameter_name)',                                                                     @parameter_value = @var,                                                                     @object_name = '$($overridevalue.object_name)',                                                                     @value_type = 'V'"     } } else # Integer or other number {     if($overridevalue.object_type -eq 20) # Project parameter     {         $updateparameterquery = "EXEC [catalog].[set_object_parameter_value]                                                                     @object_type = $($overridevalue.object_type),                                                                     @folder_name = '$($FolderName)',                                                                     @project_name = '$($ProjectName)',                                                                     @parameter_name = '$($overridevalue.parameter_name)',                                                                     @parameter_value = $($overridevalue.default_value),                                                                     @value_type = 'V'"     }     elseif($overridevalue.object_type -eq 30) # Package parameter     {         $updateparameterquery = "EXEC [catalog].[set_object_parameter_value]                                                                     @object_type = $($overridevalue.object_type),                                                                     @folder_name = '$($FolderName)',                                                                     @project_name = '$($ProjectName)',                                                                     @parameter_name = '$($overridevalue.parameter_name)',                                                                     @parameter_value = $($overridevalue.default_value),                                                                     @object_name = '$($overridevalue.object_name)',                                                                     @value_type = 'V'"      } } if($WhatIf) {     $updateparameterquery } else {     try     {         Invoke-Sqlcmd -ServerInstance $TargetServer -Database SSISDB -Query $updateparameterquery -ConnectionTimeout 5 -QueryTimeout 5 -ErrorAction Stop         Write-Output "Updated parameter $($overridevalue.parameter_name)"     }     catch     {         Write-Output "Error updating parameter $($overridevalue.parameter_name)"         Write-Output $updateparameterquery         Write-Output $error[0]     } }
 }
 }

Automating SSISDB Migration 3/4 – Copying SSIS Environments with Powershell

Part 3 of a 4-part series on automating an SSISDB migration without using the SSISDB Upgrade Wizard. Parts one and two covered exporting MSDB and SSISDB projects, after which upgrading then deploying from Visual Studio is neccessary. This post assumes this is done and the next task is to copy the environments over, as they exist outside of SSIS projects and are thus not carried with them.

To automate this we have to do a few different things;

  • Copy the environments
  • Copy environment variables
  • Copy environment references (between environment to project)
  • Copy environment-object parameter links

I created the Copy-SSISEnvironments function shown below to perform these steps. It works at folder level, taking a source/target server and foldername. Though it’s possible to directly update the internal schema tables with the required values, I’m using the built-in catalog stored procedures which wrap and sanity check the required code for safety, namely;

  • [catalog].[create_environment]
  • [catalog].[create_environment_variable]
  • [catalog].[create_environment_reference]
  • [catalog].[set_object_parameter_value]

The function supports ‘relative’ references (that is projects referencing an Environment in the same Folder), string, boolean and numeric datatypes, de and re-encrypting encrypted strings, and both project and package-level object references. It doesn not support copying ‘absolute’ references (projects referencing an Environment in a different folder) or decrypting non-string datatypes as I had no need for that in our environment. The -WhatIf switch will simply output the SQL statements it would otherwise execute, and is recommended to run before use to check the output.

Example – Copy-SSISEnvironments -SourceServer SQLSSIS2014 -TargetServer SQLSSIS2017 -FolderName MySSISFolder

Github link to the Powershell-SSIS repo I’ve added this to.

function Copy-SSISEnvironments
{
param
(
    [Parameter(position=0, mandatory=$true)][string]$SourceServer,
    [Parameter(position=1, mandatory=$true)][string]$TargetServer,
    [Parameter(position=2, mandatory=$true)][string]$FolderName,
    [Parameter(position=3, mandatory=$false)][switch]$WhatIf
)

<#
.SYNOPSIS
Copies environments and references from one SSISDB folder to the same folder on another server

.DESCRIPTION
Copies environments - including variables, project references and object-variable links - from a source SSISDB folder to the same folder on a target server.
Works with string, numeric, boolean and encrypted string variables and relative project references.
Does not copy absolute references.

.PARAMETER SourceServer
Specifies the source SQL instance name

.PARAMETER TargetServer
Specifies the target SQL instance name

.PARAMETER FolderName
Specifies the SSISDB folder to copy the environment from and to. Case sensitive.

.PARAMETER WhatIf
Outputs generated SQL statements only - does not execute them or make any other changes.

.OUTPUTS
No direct outputs from function - updates target server SSISDB folder

.EXAMPLE
PS> Copy-SSISEnvironments -SourceServer SQLSSIS2014 -TargetServer SQLSSIS2017 -FolderName DailyETL

Copies environment data from SSISDB folder 'DailyETL' on server SQLSSIS2014 to server SQLSSIS2017
#>

Import-Module SqlServer

### Check that folder exists on both servers
$checkexistsquery = "SELECT 1 FROM [catalog].[folders] f
                     WHERE f.[name] = '$($FolderName)'"

try
{
    $check = Invoke-Sqlcmd -ServerInstance $SourceServer -Database SSISDB -Query $checkexistsquery -ConnectionTimeout 5 -QueryTimeout 5 -ErrorAction Stop
}
catch
{
    Write-Output "Error connecting to source server."
    Write-Output $error[0]
    return
}


if(!$check)
{
    Write-Output "Error: Folder $($FolderName) doesn't exist on source server."
    return
}

try
{
    $check = Invoke-Sqlcmd -ServerInstance $TargetServer -Database SSISDB -Query $checkexistsquery -ConnectionTimeout 5 -QueryTimeout 5 -ErrorAction Stop
}
catch
{
    Write-Output "Error connecting to target server."
    Write-Output $error[0]
    return
}

if(!$check)
{
    Write-Output "Error: Folder $($FolderName) doesn't exist on target server."
    return
}



$source_folder_id = 0
$target_folder_id = 0

$GetFolderIDQuery = "SELECT folder_id
                    FROM [catalog].folders
                    WHERE [name] = '$($FolderName)'"

# Get folder_id of source and target folders
$source_folder_id = (Invoke-Sqlcmd -ServerInstance $SourceServer -Database SSISDB -Query $GetFolderIDQuery -ConnectionTimeout 5 -QueryTimeout 5 -ErrorAction Stop).folder_id
$target_folder_id = (Invoke-Sqlcmd -ServerInstance $TargetServer -Database SSISDB -Query $GetFolderIDQuery -ConnectionTimeout 5 -QueryTimeout 5 -ErrorAction Stop).folder_id

$GetEnvironmentQuery = "SELECT environment_id, [name], [description]
                            FROM [catalog].environments
                            WHERE folder_id = '$($source_folder_id)'"

# Get environments from the source folder
$envs = (Invoke-Sqlcmd -ServerInstance $SourceServer -Database SSISDB -Query $GetEnvironmentQuery -ConnectionTimeout 5 -QueryTimeout 5 -ErrorAction Stop)


# Get environment references 
$GetEnvironmentRefsQuery = "SELECT p.project_id, p.[name] AS 'ProjectName', er.reference_type, er.environment_name
                            FROM [catalog].projects p
                            INNER JOIN [catalog].environment_references er
	                            ON p.project_id = er.project_id
                            WHERE p.folder_id = $($source_folder_id)"

try
{
    $env_refs = Invoke-Sqlcmd -ServerInstance $SourceServer -Database SSISDB -Query $GetEnvironmentRefsQuery -ConnectionTimeout 5 -QueryTimeout 5 -ErrorAction Stop
}
catch
{
    Write-Output "Error retrieving environment references"
    $GetEnvironmentRefsQuery
    Write-Output $error[0]
}

 
foreach($env in $envs)
{
    # Create environment on target server
    $CreateNewEnvironmentQuery = "EXEC [SSISDB].[catalog].[create_environment] @folder_name = '$($FolderName)', @environment_name = '$($env.name)', @environment_description = '$($env.description)'"

    if($WhatIf)
    {
        Write-Output $CreateNewEnvironmentQuery
    }
    else
    {
        try
        {
            Invoke-Sqlcmd -ServerInstance $TargetServer -Database SSISDB -Query $CreateNewEnvironmentQuery -ConnectionTimeout 5 -QueryTimeout 5 -ErrorAction Stop
            Write-Output "Created environment $($env.name)"
        }
        catch
        {
            Write-Output "Error creating environment"
            Write-Output $CreateNewEnvironmentQuery
            Write-Output $error[0]

            return
        }
    }

    # Get environment variables
    $GetEnvironmentVarsQuery = "SELECT variable_id, [name], sensitive, [type], [description], [value]
                                FROM [catalog].environment_variables
                                WHERE environment_id = $($env.environment_id)"

    $env_vars = (Invoke-Sqlcmd -ServerInstance $SourceServer -Database SSISDB -Query $GetEnvironmentVarsQuery -ConnectionTimeout 5 -QueryTimeout 5 -ErrorAction Stop)

    # Loop over environment variables
    foreach($env_var in $env_vars)
    {
        if($env_var.type -eq 'String')
        {
            # If variable is sensitive string, pull the encrypted value from [internal] table and decrypt it
            if($env_var.sensitive -eq 1)
            {
                $getencryptedvaluequery = "SELECT [internal].[get_value_by_data_type](DECRYPTBYKEYAUTOCERT(CERT_ID(N'MS_Cert_Env_' + CONVERT(NVARCHAR(20), [environment_id])), NULL, [sensitive_value]), [type]) AS DecryptedValue
                                            FROM [SSISDB].[internal].[environment_variables]
                                            WHERE variable_id = $($env_var.variable_id)"

                try
                {
                    $decryptedvalue = (Invoke-Sqlcmd -ServerInstance $SourceServer -Database SSISDB -Query $getencryptedvaluequery -ConnectionTimeout 3 -QueryTimeout 5 -ErrorAction Stop).DecryptedValue

                    $CreateEnvironmentVarQuery = "EXEC [SSISDB].[catalog].[create_environment_variable] @environment_name = '$($env.name)',
	                                                                                                    @folder_name = '$($FolderName)',
	                                                                                                    @variable_name = '$($env_var.name)',
	                                                                                                    @Sensitive = 1,
	                                                                                                    @data_type = '$($env_var.type)',
	                                                                                                    @description = '$($env_var.description)',
	                                                                                                    @value = N'$($decryptedvalue)'"
                }
                catch
                {
                    Write-Output "Failed to retrieve encrypted variable value"
                    $getencryptedvaluequery
                    Write-Output $error[0]
                }

            } 
            else
            {
                $CreateEnvironmentVarQuery = "EXEC [SSISDB].[catalog].[create_environment_variable] @environment_name = '$($env.name)',
	                                                                                                @folder_name = '$($FolderName)',
	                                                                                                @variable_name = '$($env_var.name)',
	                                                                                                @Sensitive = 0,
	                                                                                                @data_type = '$($env_var.type)',
	                                                                                                @description = '$($env_var.description)',
	                                                                                                @value = N'$($env_var.value)'"
            }
        }
        elseif($env_var.type -eq 'Boolean')
        {
            if($env_var.value -eq 'True')
            {
                $bool_value = 1
            }
            else
            {
                $bool_value = 0
            }

            $CreateEnvironmentVarQuery = "EXEC SSISDB.[catalog].[create_environment_variable] @environment_name = '$($env.name)',
	                                                                                            @folder_name = '$($FolderName)',
	                                                                                            @variable_name = '$($env_var.name)',
	                                                                                            @Sensitive = $([int]$env_var.sensitive),
	                                                                                            @data_type = '$($env_var.type)',
	                                                                                            @description = '$($env_var.description)',
	                                                                                            @value = $($bool_value)"

        }
        else
        {
            $CreateEnvironmentVarQuery = "EXEC SSISDB.[catalog].[create_environment_variable] @environment_name = '$($env.name)',
	                                                                                            @folder_name = '$($FolderName)',
	                                                                                            @variable_name = '$($env_var.name)',
	                                                                                            @Sensitive = $([int]$env_var.sensitive),
	                                                                                            @data_type = '$($env_var.type)',
	                                                                                            @description = '$($env_var.description)',
	                                                                                            @value = $($env_var.value)"
        }
        

        if($WhatIf)
        {
            Write-Output $CreateEnvironmentVarQuery
        }
        else
        {
            try
            {
                Invoke-Sqlcmd -ServerInstance $TargetServer -Database SSISDB -Query $CreateEnvironmentVarQuery -QueryTimeout 5 -ConnectionTimeout 5 -ErrorAction Stop
                Write-Output "Created variable $($env_var.name)"
            }
            catch
            {
                Write-Output "Error creating environment variable."
                Write-Output $CreateEnvironmentVarQuery
                Write-Output $error[0]
            }
        }
    }

    # Check if a reference exists for this environment, then for projects in target folder
    foreach($env_ref in $env_refs)
    {
        if($env_ref.reference_type -eq 'A')
        {
            Write-Warning "Warning - Absolute reference detected. Add manually. Project name $($env_ref.ProjectName)"
        }
        elseif($env_ref.environment_name -eq $env.name)
        {
            $CheckProjectExistsQuery = "SELECT 1 AS 'Result' WHERE EXISTS (SELECT [name]
                                                            FROM [catalog].projects
                                                            WHERE folder_id = $($target_folder_id)
                                                            AND name = '$($env_ref.ProjectName)')"

            #### Add reference for $env_ref.ProjectName

            try
            {
                $exists = (Invoke-SqlCmd -ServerInstance $TargetServer -Database SSISDB -Query $CheckProjectExistsQuery -ConnectionTimeout 5 -QueryTimeout 5 -ErrorAction Stop).Result

                if($exists -eq 1)
                {
                    # Add environment reference for this project
                    $CreateEnvironmentRefQuery = "EXEC [SSISDB].[catalog].[create_environment_reference] @environment_name = '$($env.name)',
														    @project_name = '$($env_ref.ProjectName)',
														    @folder_name = '$($FolderName)',
														    @reference_type = R,
                                                            @reference_id = NULL"
                    
                    if($WhatIf)
                    {
                        Write-Output $CreateEnvironmentRefQuery
                    }
                    else
                    {
                        try
                        {
                            Invoke-Sqlcmd -ServerInstance $TargetServer -Database SSISDB -Query $CreateEnvironmentRefQuery -ConnectionTimeout 5 -QueryTimeout 5 -ErrorAction Stop
                            Write-Output "Created reference between $($env_ref.ProjectName) and $($env.name)"


                            
                            #### Set object parameters to use environment variables where linked

                            $getobjectparamsquery = "SELECT object_type, [object_name], parameter_name, referenced_variable_name
                                        FROM catalog.object_parameters
                                        WHERE project_id = $($env_ref.project_id)
                                        AND value_type = 'R'"

                            $object_params = Invoke-Sqlcmd -ServerInstance $SourceServer -Database SSISDB -Query $getobjectparamsquery -ConnectionTimeout 3 -QueryTimeout 5 -ErrorAction Stop

                            foreach($object_param in $object_params)
                            {
                                if($object_param.object_type -eq 20) # Project parameter
                                {
                                    $setobjectparamquery = "EXEC [catalog].[set_object_parameter_value]
                                                                    @object_type = $($object_param.object_type),
                                                                    @folder_name = '$($FolderName)',
                                                                    @project_name = '$($env_ref.ProjectName)',
                                                                    @parameter_name = '$($object_param.parameter_name)',
                                                                    @parameter_value = '$($object_param.referenced_variable_name)',
                                                                    @value_type = 'R'"    
                                }
                                elseif($object_param.object_type -eq 30) # Package parameter
                                {
                                    $setobjectparamquery = "EXEC [catalog].[set_object_parameter_value]
                                            @object_type = $($object_param.object_type),
                                            @folder_name = '$($FolderName)',
                                            @project_name = '$($env_ref.ProjectName)',
                                            @parameter_name = '$($object_param.parameter_name)',
                                            @parameter_value = '$($object_param.referenced_variable_name)',
                                            @object_name = '$($object_param.object_name)',
                                            @value_type = 'R'"  
                                }

                                try
                                {
                                    Invoke-Sqlcmd -ServerInstance $TargetServer -Database SSISDB -Query $setobjectparamquery -ConnectionTimeout 3 -QueryTimeout 5 -ErrorAction Stop
                                    Write-Output "Linked parameter $($object_param.parameter_name) to env var $($object_param.referenced_variable_name)"
                                }
                                catch
                                {
                                    Write-Output "Error linking parameter."
                                    $setobjectparamquery
                                    Write-Output $error[0]
                                }

                            }

                        }
                        catch
                        {
                            Write-Output "Error creating environment reference."
                            $CreateEnvironmentRefQuery
                            Write-Output $error[0]
                        }
                    }
                }
            }
            catch
            {
                Write-Output $CheckProjectExistsQuery
                Write-Output $error[0]
            }


            if($WhatIf)
            {
                #### Set object parameters to use environment variables where linked

                $getobjectparamsquery = "SELECT object_type, [object_name], parameter_name, referenced_variable_name
                                        FROM catalog.object_parameters
                                        WHERE project_id = $($env_ref.project_id)
                                        AND value_type = 'R'"

                $object_params = Invoke-Sqlcmd -ServerInstance $SourceServer -Database SSISDB -Query $getobjectparamsquery -ConnectionTimeout 3 -QueryTimeout 5 -ErrorAction Stop

                foreach($object_param in $object_params)
                {
                    if($object_param.object_type -eq 20)
                    {
                        $setobjectparamquery = "EXEC [catalog].[set_object_parameter_value]
                                                        @object_type = $($object_param.object_type),
                                                        @folder_name = '$($FolderName)',
                                                        @project_name = '$($env_ref.ProjectName)',
                                                        @parameter_name = '$($object_param.parameter_name)',
                                                        @parameter_value = '$($object_param.referenced_variable_name)',
                                                        @value_type = 'R'"    
                    }
                    elseif($object_param.object_type -eq 30)
                    {
                        $setobjectparamquery = "EXEC [catalog].[set_object_parameter_value]
                                @object_type = $($object_param.object_type),
                                @folder_name = '$($FolderName)',
                                @project_name = '$($env_ref.ProjectName)',
                                @parameter_name = '$($object_param.parameter_name)',
                                @parameter_value = '$($object_param.referenced_variable_name)',
                                @object_name = '$($object_param.object_name)',
                                @value_type = 'R'"  
                    }

                    $setobjectparamquery
                }
            }

        }

    }

}

}

Automating SSISDB Migration 2/4 – Exporting SSISDB Projects to .ispac with Powershell

Continuing from the first step of a legacy SSIS migration project – exporting MSDB packages – the next is to export SSISDB projects. As I mentioned in the previous post, Microsoft has included a handy wizard to do this for you since SQL 2016, however it might not be appropriate for you to use, as it wasn’t at my company – we were ditching a lot of of the existing projects, editing some of the others, and wanted a clean slate.

SSISDB projects, unlike their MSDB ancestors, are relatively sophisticated beasts and exported as .ispac (project) files. They can be exported easily via SSMS one at a time – but if you need to export hundreds, it’s not as simple as dumping out a VARBINARY conversion of column data.

Thanks to the SSIS .NET API it’s still pretty easy, though. We can access the SSIS instance, enumerate the folders and projects, then call the GetProjectBytes() function on each to get the .ispac data.

The below Powershell function from the Powershell SSIS repo I’m working on takes an instance name and output directory and outputs all of the projects as .ispac files, creating a subfolder structure in the output directory mirroring the SSISDB folders.

For example, for an SSISDB with three folders – QuarterlyReports, ChurnDataPointlessly, AwfulQueryGreatestHits, each with a Project called IntegrationServicesProject1 – and given an output dir of D:\DBA\, the result would be the following subfolders and files created;

D:\DBA\QuarterlyReports\IntegrationServicesProject1.ispac
D:\DBA\ChurnDataPointlessly\IntegrationServicesProject1.ispac
D:\DBA\AwfulQueryGreatestHits\IntegrationServicesProject1.ispac

The script assumes the user can authenticate via AD and has permissions to create the filesystem artefacts.

Github link

function Export-SSISProjectSSISDB
 {
 param
 (
     [Parameter(position=0, mandatory=$true)][string]$Instance,
     [Parameter(position=1, mandatory=$true)][string]$OutputDir
 )
 if(!(Test-Path $OutputDir))
 {
     Write-Output "Error - invalid path specified in OutputDir"
     
     return
 }
 $testquery = "SELECT COUNT(*) AS 'Result' FROM sys.databases WHERE name = 'SSISDB'"
 try
 {
     $result = (Invoke-Sqlcmd -ServerInstance $Instance -Query $testquery -ConnectionTimeout 5 -QueryTimeout 5 -ErrorAction Stop).Result
     
     if($result -eq 0)
     {
          Write-Output "Error - no SSISDB present on instance."
    
          return
     }
 }
 catch
 {
     Write-Output "Error - failure connecting to instance"
     return
 }
 
 [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.Management.IntegrationServices") | Out-Null
 
 $SSISnamespace = "Microsoft.SqlServer.Management.IntegrationServices"
 
 $connstring = "Data source=$($Instance);Initial Catalog=master;Integrated Security=SSPI;"
 $sqlconn = New-Object System.Data.SqlClient.SqlConnection $connstring
 
 $SSIS = New-Object $SSISnamespace".IntegrationServices" $sqlconn

 $catalog = $SSIS.Catalogs["SSISDB"]
 foreach($folder in $catalog.Folders)
 {
     Set-Location -Path $outputdir

     New-Item -ItemType Directory -Name $folder.Name | Out-Null
  
     $folderpath = $outputdir + "\" + $folder.Name
 
     Set-Location -path $folderpath

     $projects = $folder.Projects

     if($projects.Count -gt 0)
     {
          foreach($project in $projects)
          {
               $projectpath = $folderpath + "\" + $project.Name + ".ispac"

               Write-Host "Exporting to $($projectpath) ...";
     
               [System.IO.File]::WriteAllBytes($projectpath, $project.GetProjectBytes())
          }
     }
 }
 <#
 .SYNOPSIS
 Exports all SSIS projects from an SSISDB database to a specified output directory
 .DESCRIPTION
 Retrieves all SSIS projects in .ispac format from an SSISDB database to a specified output directory, creating SSISDB folder structure.
 .PARAMETER Instance
 Specifies the SQL instance name
 .PARAMETER OutputDir
 Specifies the full output directory for SSISDB Folders/Projects to be exported to.
 .OUTPUTS
 No direct outputs from fuction - writes .ispac files.
 .EXAMPLE
 PS> Export-SSISProjectSSISDB -Instance SQLSSIS2014 -OutputDir "D:\DBA\SSIS\Export"
 Exports all SSIS projects from instance SQLSSIS2014 as .ispac files to SSISDB Folder-named subfolders created in D:\DBA\SSIS\Export 
 #>
 }