Skip to main content
Version: 3.8 (Latest)

Fusion Installation

Deployment path

This guide is for Fusion with Azure Data Explorer (ADX) installations. Use the separate Fusion in Fabric installation guide for Fabric deployments. Choose one installation path per customer environment.

Prerequisites​

Ensure you have the permission, the User executing the terraform should have owner rights to the RG (mainly to add IAM security).

Download and install Terraform. https://www.terraform.io/downloads.html (For Fusion v3.8, Terraform version 1.14.3 is recommended.)

Configuration instructions​

  1. Get all the various configuration information required for an install. This is outlined in example_consumer/main.tf and includes the following:
VariableDescriptionSample Data
deploymentTypeType of deployment used to ingest data: must be one of "Elevate", "Ignition", or ""."Elevate"
resourceGroupNameName of the resource group to deploy to. This value should not contain uppercase characters."rg-fusion"
companyNameName of the company. This value must be globally unique and not contain uppercase characters."company-name"
environmentTypeName of the environment type. This must be one of the following values (in all lowercase):
  • dev
  • test
  • prod
  • "dev" #Development Environment
  • "test" #Test Environment
  • "prod" #Production Environment
tenantIdUnique identifier for the tenant of the resource group
subscriptionIdUnique identifier for the subscription of the resource group
essentialsAdminObjectIdUnique identifier for the "Admin-Group-Name" Azure Active Directory group
adminGroupName readWriteGroupName readGroupNameThe specified Azure Active Directory groups from the previous step. These groups are used for SQL Server access and role-based access to the model hierarchy UI.
  • adminGroupName = "Admin-Group-Name"
  • readWriteGroupName = "ReadWrite-Group-Name"
  • readGroupName = "Read-Group-Name"
sqlModelsUsername sqlModelsPasswordDefault SQL user account.sqlModelsUsername = "username" sqlModelsPassword = "password"
clientId clientSecretClient ID: Leave blank if appRegistrationCount = 1. If appRegistrationCount = 0, you may specify the client ID of an existing App Registration. That registration must already be configured with correct API permissions, reply URIs, etc., since Terraform will not manage it. Client Secret: A user-configured plain-text password. A randomly generated Base64-encoded value is recommended. If left blank, it will be auto-generated.
licenseKeyThe license key for the product. Details for creating a license key can be found here: Creating a License Key for a Customer
tagControllerCountIf you're deploying multiple Elevates per instance of Fusion, the tagControllerCount can be greater than 1. Set to 0 for Ignition or other non-Elevate deployments.tagControllerCount = 1
gapFillCountMust be 0 as Gap Fill is not currently a supported feature.gapFillCount = 0
appRegistrationCountSet to 1 to create a new application registration. Set to 0 to use an existing app reg; see clientId.appRegistrationCount = 0
duplicatorCountSet to 1 to deploy a DPP function app, which is necessary for Elevate, Ignition, and some other deployments. If it is set to 0, a connection from the IoT hub to ADX (or whatever) must be made manually after deployment.duplicatorCount = 1
kqlModeFor this ADX installation guide, set KQL mode to adx.kqlMode = "adx"
front_door_skuSet to "none" to not install Azure Front Door. Defaults to "classic", but this sku is no longer supported.front_door_sku = "none"
tagControllerDevSqlServerWhen environmentType is "dev", this specifies whether to use a SQLite (false) or a SQL Server (true) database. Defaults to false (SQLite).tagControllerDevSqlServer = true
  1. Create a subdirectory in deployments/ under /dev or /prod and copy providers.tf, state.tf, and versions.tf from example_consumer/ to the new directory.
  2. Fill in the required information in state.tf for remote state storage. This will require a SAS token to be created.
VariableDescriptionSample Values
resource_group_nameThe name of the resource group where the backend state storage blob lives (this is not the same as the resource group in step 1)
storage_account_nameThe name of the storage account where the state blob will be stored
container_nameThe name of the container where the state blob will be stored
sas_tokenA Shared Access Signature token to provide access to the state storage containersp=rwl&st=...
keyThe name of the state file for this deploymentprod.terraform.tfstate
tenant_idUnique identifier for the tenant of the resource group of the storage account.A guid
subscription_idUnique identifier for the subscription of the resource group of the storage accountA guid

Direct Invocation (Follow this if you are not storing config in a key vault)​

  1. Copy example_consumer/main.tf to the install directory and rename it to something descriptive (e.g., main.tf, product.tf, etc.)
  2. Fill in the respective configuration information in the inputs described in your new main file.

Pre-Installation Instructions​

  1. If you are not already using the module version of the terraform scripts run fusion_pre_install.ps1
Finding which invocation is being used

If you aren't sure which version of the terraform the install has used, you can run terraform state list and check if the the names of resources. If they are prefixed with module.<someModuleName> then the installation has already been updated to a new terraform version and you can skip running fusion_pre_install.ps1

  1. Ensure that the following Azure Active Directory groups have been created. (Note you can specify your own group names and configure them in the deployment configuration
  • Admin
  • ReadWrite
  • Read

Installation Instructions​

App Registration Pre-Install​

Prerequisites​

  1. Create a resource group.
  2. The following Azure AD groups have been created (Note: groups can follow customer naming conventions).
  • Read
  • ReadWrite
  • Admin
  1. Create a service principal user with the following settings.
  • Owner role for resource group created previously.
  • Application administrator
  • Directory reader
  • Directory writer
  • Privileged role administrator

Below is a sample command to create the service principal user. Note: This will create the service principal but it will still need the above settings from step 3.

az ad sp create-for-rbac --name "sp-fusion-deployment" --role owner --scopes /subscriptions/<Subscription_ID>/resourceGroups/<Resource_Group>
Note

Note the ServicePrincipalApplicationID and ServicePrincipalPassword as they are needed later.

  1. The user running the terraform script will need to have rights to create a storage account resource.

Software​

Contact your Fusion team for the app-registration-pre-install.zip which contains the terraform files needed.

App Registration Pre-Install zip file

When creating the app-registration-pre-install.zip, it must contain both the app-registration-pre-install and cloud-scaffolding folders.

Run Script

  1. Open an Azure Cloud Shell window either by: a. Clicking on the following link: https://shell.azure.com/ b. Clicking on the Cloud Shell icon in your Azure Portal.

  2. You may be prompted to create a storage resource. If so, click on ‘Show advanced settings’ and select the target resource group names and add names for the storage account and file share.

  3. Login with the service provider credentials changing the ServicePrincipalApplicationID and ServicePrincipalPassword to be those of the service provider and the TenantId to be the tenant id.

az login --service-principal -u " ServicePrincipalApplicationId" -p "ServicePrincipalPassword" --tenant "TenantId"
  1. Set the following environment variables substituting the correct values for your system and service provider.
$env:ARM_TENANT_ID = "TenantId "
$env:ARM_SUBSCRIPTION_ID = "SubscriptionId"
$env:ARM_CLIENT_ID = "ServicePrincipalApplicationId"
$env:ARM_CLIENT_SECRET = “ServicePrincipalPassword"

  1. Upload the app-registration-pre-install.zip to the Azure CLI window.

  2. Extract the files.

unzip app-registration-pre-install.zip
  1. Edit the main.tf file in the app-registration-pre-install/example_caller directory. You can edit the file in Azure CLI by clicking the in the PowerShell menu and using the directory tree on the left to find the file. Double click on the file to open it for editing on the right-hand side. To save the file, right click on the edit pane and select save and or quit.

Change the following to match your environment:

  • resourceGroupName - This should be for the resource group created previously.
  • companyName - This should be lowercase and no more than 20 characters.
  • environmentType - should be dev, test or prod
  • tagControllerCount = 1
  • readGroupName=”Read-Group-Name”
  • readWriteGroupName=“ReadWrite-Group-Name”
  • adminGroupName=”Admin-Group-Name”
  • tenantId
  • subscriptionId
  1. Change directory to /app-registration-pre-install/example_caller.
Note

This path may be different if the files were unzipped to a different path. i.e. cd app-registration-pre-install/example_caller

  1. Delete any files other than main.tf and providers.tf in this directory.
  2. From the Powershell command prompt, type terraform init.
  3. Type terraform plan to see what terraform actions will be performed.
  4. To begin the installation, type terraform apply. Type yes if prompted to implement the changes.
  5. This should complete with no errors. If not, contact your Fusion team member for help.
  6. From the Azure portal, check the resource group and verify that a key vault was created.
  7. Within the key vault, verify that the service principal is listed as an Owner role assignment.
  8. Check app registrations to see the newly created one. The owner of the app registration should be the service principal.
  9. To continue with the Fusion deployment, the Fusion team member will need to be added to the key vault created as an owner. This can be removed after deployment if needed.

Connections​

Run the following commands in PowerShell to connect to Azure:

$env:ARM_TENANT_ID = **"<Tenant ID>"**
$env:ARM_SUBSCRIPTION_ID = **"<Subscription ID>"**
az login --tenant $env:ARM_TENANT_ID
az account set --subscription $env:ARM_SUBSCRIPTION_ID
Connect-AzureAD -TenantId $env:ARM_TENANT_ID
Connect-AZAccount -TenantId $env:ARM_TENANT_ID
Set-AzContext -Subscription $env:ARM_SUBSCRIPTION_ID -Tenant $env:ARM_TENANT_ID

First run​

If this is the first time this terraform is being executed for a given deployment the first_run variable should be set to true. This is only for the first run of an installation, and is not required for updates. If first_run=true, do not use kqlMode="migration"; use kqlMode="adx" or kqlMode="fabric" for initial deployment.

It's recommended to set the first_run variable in the command line invocation i.e. terraform {plan, apply} -var first_run={true, false}, but this can also be set in your main tf file for that deployment. The following install instructions assume the first_run flag is set on the command line, but -var first_run=true can be removed if set elsewhere.

Install instructions​

  1. Run terraform init while in the example consumer file location
  2. Run terraform plan -var first_run=true to ensure that the plan looks correct. Note: Use -var first_run=false if Fusion has been deployed before.
  3. Run terraform apply -var first_run=true and double check that the output is correct. If so type yes and run the plan.
  4. If any errors saying something along the lines of "A resource with the ID ... already exists" then run the following command for each resource that errored, and then return to step 3. The message will say with module.<module_name>.
  • terraform import <module_path> <resource_id>
  1. If this apply was run with first_run=true, then apply the terraform again with first_run=false. For example, terraform apply -var first_run=false.
Terraform Target resource

If during deployment one of the resources was not deployed, such as the PowerShell scripts being run against the database, terraform can run only that specific resource with the following command:

.\terraform apply -target="resourcetype.resourcename" eg. terraform apply -target="null_resource.createSqlUser"