Azure Deployment Guide

Contents 1. Building an EFT Gold Image * Part I: Creating a VM on Azure * Part II: Installing EFT * Part III: VM Prep * Part IV: Firewall Configuration * Part V: Setting up the Registry * Part VI: Misc Tasks * Part VII: Cleanup and Sysprep 1. Capturing the VM Image * Using Azure Powershell * Testing the Captured Image 1. Deployment * Part 0: Preliminaries * Part I: Storage Account for Azure Files and Bootstrap Script * Part II. Configuring the Bootstrap script * Editing the bootstrap script * Uploading the bootstrap script * Part III: Scale Set Template * Part IV: Deploying the template 1. Azure SQL Server and EFT ARM * Part I: Creating an Azure SQL Database * Part II: Setting up the Database * Part III: Setting up EFT to use Azure SQL Server 1. Azure Active Directory Domain Services * Enabling Domain Services * If your VMs are on a different Azure Virtual Network from your AADDS * If your VMs are not on an Azure Virtual Network at all * Misc AADDS Notes * Using Secure LDAP (LDAPS) 1. Deployment from Shared Image 1. Scaling with Draining

Building an EFT Gold Image

The following sections describe how to create an EFT Gold Image. If you have been supplied with a pre-built gold image, then you can skip this section and proceed to deployment.

Part I: Creating a VM on Azure

  1. Launch VM in appropriate resource group (click on resource group link, then +Add)
  2. Select VM size; standard F1S with SSD is adequate for testing
  3. Select the region. South Central U.S. is preferred to minimize latency.
  4. Select unmanaged disk. (Currently unable to obtain a VHD and a blob URI to supply to our scale set template using managed ones)

Part II: Installing EFT

  1. Copy installer to VM
  2. Begin installation
  3. Use “admin” and “myAdmin123” when prompted for the Administrator username and password
  4. Install EFT as an Active-active cluster
  5. Specify that this is the first node when prompted by the dialog box
  6. Use c:\interim for the shared folder (the shared folder doesn’t necessarily exist yet)

Part III: VM Prep

  1. Download Sysinternal’s VolumeID tool to the VM
  2. Unzip executable to Windows Desktop
  3. Shift + Right Click on the Desktop
  4. Select “Open command window here”
  5. Run command Volumeid64 c: fc5a-78bd (the third digit must be 5 - the others are arbitrary) (TODO: PM is aware of this general purposefully-obfuscated approach, but confirm they buy into the specifics.)
  6. Delete VolumeID utility executable
  7. Delete VolumeID utility zip file
  8. Reboot the machine so the volume id change will take effect.
  9. Copy AmazonMeteredSubstringCfg.exe to EFT’s installation directory on the VM (“C:\Program Files (x86)\Globalscape\EFT Server Enterprise” by default)
  10. Run AmazonMeteredSubstringCfg.exe on the VM
  11. Enter “volume” when prompted for the encrypted file name
  12. Enter “encrypt” when asked for the operation to perform
  13. Delete AmazonMeteredSubstringCfg.exe

Part IV: Firewall Configuration

  1. Open Powershell or a Command Prompt as Administrator
  2. Run gpedit.msc
  3. Expand “Computer Configuration” -> “Windows Settings” -> “Security Settings” -> “Windows Firewall with Advanced Security” -> “Windows Firewall with Advanced Security” - “Local Group Policy Object”
  4. Right-click on Inbound Rules
  5. Select “New Rule” from context menu
  6. Select “Port” radio button in the New Inbound Rule Wizard Dialog which appears
  7. Click “Next >” button
  8. Select “TCP” radio button
  9. Select “Specific local ports” radio button
  10. Enter “80, 443” (no quotation marks) in the text field adjacent to “Specific local ports”.
  11. Click “Next >” button
  12. Select “Allow the connection” radio button
  13. Check Domain, Private, and Public checkboxes when asked to which the rule applies
  14. Click “Next >”
  15. Enter in arbitrary name for Name
  16. Click Finish
  17. Close gpedit.msc

Part V: Setting up the Registry

  1. Open the 32-bit version of Powershell, “Windows Powershell (x86)”, as Administrator
  2. Run the command regedit.exe
  3. Expand “HKEY_LOCAL_MACHINE” -> “SOFTWARE” -> “GlobalSCAPE Inc.” -> “EFT Server 7.0”
  4. Right-click on “EFT Server 7.0” folder icon
  5. Select “New” -> “Key” from context menu
  6. Rename newly created key “HA Settings”
  7. Right click on newly created key “HA Settings”
  8. Select “New” -> “String Value”
  9. Rename newly created String Value to “SharedConfigurationPath”
  10. Leave value of String Value “SharedConfigurationPath” empty
  11. Close regedit.exe

Part VI: Misc Tasks

  1. Install SQL Server Native Client 11.0 on local machine, as part of building Gold Image, if using the Audit and Reporting Module (and if it is not already installed)

Part VII: Cleanup and Sysprep

WARNING: Make sure you have done all the setup you want to do before proceding. You will not be able to start the VM again once it is generalized / sysprepped.

  1. Delete the c:\interim directory and all its contents
  2. Open Powershell (64 bit) as Administrator
  3. Run command Remove-WindowsFeature MSMQ
  4. Reboot VM
  5. Open a command prompt as Administrator
  6. Run the command %windir%\\System32\\Sysprep\\sysprep.exe
  7. Select “Enter System Out-of-Box Experience (OOBE) for System Cleanup Action
  8. Check the “Generalize” checkbox
  9. Select “Shutdown” for Shutdown Options
  10. Click “OK”
  11. Wait for the VM to finish sysprepping (you will lose your connection to it when it shuts down)

Capturing the VM Image

Using Azure Powershell

(Adapted from https://docs.microsoft.com/en-us/azure/virtual-machines/windows/capture-image)

  1. Select your VM in the Azure Resource Management Console
  2. Click the “Stop” button
  3. Wait for your VM to stop
  4. Open Azure Powershell
  5. Use the cd command to cd to a directory in which you have full permissions
  6. Run command Login-AzureRmAccount
  7. Enter the Azure Credentials for the account which owns the VM you created
  8. Run command Get-AzureRmSubscription
  9. Copy the “Id” value to the clipboard using your mouse
  10. Run command Select-AzureRmSubscription -SubscriptionId <paste id here>
  11. Run command Set-AzureRmVm -ResourceGroupName <name of your VM's resource group> -name <name of your VM> -Generalized
  12. Run command Save-AzureRmVMImage -ResourceGroupName <name of your resource group> -Name <name of your VM> -DestinationContainerName <name of a container to store your image in on Azure> -VHDNamePrefix <string to prefix your image's name with> -Path .\<Arbitrary name for file to receive>.json
  13. Open the Storage account with the “system” and “vhds” containers
  14. Open “system” -> “Microsoft.Compute” -> “Images” -> Name your provided for container
  15. Click on the *.vhd file
  16. Record the URI/URL from the pane that opens to the right (you will use this later for a Deployment Template parameter)

Testing the Captured Image

If desired, use the 101-vm-from-user-image quick start template to smoke test the captured image independent of the scale set template, bootstrap script, etc.

Deployment

The following sections describe how to deploy a Gold EFT Image using the scale set template.
## Part 0: Preliminaries 1. Create a new resource group - this will be where your Deployment Template deploys into, and will be referenced by the bootstrap script.

Part I: Storage Account for Azure Files and Bootstrap Script

  1. Create a standard (non-premium) storage account in a new resource group, and in the same region as your EFT Gold image
  2. Create a Shared Folder in Azure Files in the storage account (Example name is ‘gsbshared’)
  3. Click on the “Overview” blade for your Azure Files storage account
  4. Record the Storage Account Name, Shared Folder name, and the “File Service Endpoint” from your Azure Files overview pane
  5. Click on the Shared Folder
  6. Click “Connect”
  7. Copy the Windows connection string to a text file for future reference
  8. Create a BLOB container within the storage account
  9. Create a private container/directory named “scripts” inside the BLOB storage
  10. Record your “Primary blob service endpoint” from the BLOB overview pane
  11. Click on the “Storage Accounts” blade again, from the Azure dashboard
  12. Click on the “Access Keys” blade
  13. Record the first key

Part II. Configuring the Bootstrap script

Editing the bootstrap script

WARNING: DO NOT CHANGE $EftAdminName, $EftAdminPass, $b64AdminNameAndPass, or $azureConfigFilename

  1. Open the bootstrap script, presently named AzureBootstrap.ps1, in an editor
  2. Set $SharedFolderStorageAcct to your Azure Files storage account name
  3. Set $SharedFolderPass to your shared folder’s access key, which is the last, contiguous part of the connection string
  4. Set $SharedFolder to the name of your Shared Folder
  5. Set $scaleSetResourceGroup to the name of the resource group you created for your Deployment Template to deploy into
  6. Set $scaleSetName to something arbitrary (i.e: “gsbvmssf”)
  7. Set $armServerName, $armDatabaseName, $armUsername, and $armPassword as necessary to connect EFT Arm to a database
  8. Set $ldapServer, $ldapUser, and $ldapPassword as necessary to connect EFT to an Active Directory

Uploading the bootstrap script

  1. Upload the edited AzureBootstrap.ps1 bootstrap script to BLOB storage in the “scripts” directory created earlier.

Part III: Scale Set Template

  1. Click on the “More Services >” blade on the Azure dashboard
  2. Search for “Templates”
  3. Click on the “Templates” blade
  4. Click “+ Add” template
  5. Give the template an arbitrary name and description
  6. Click OK
  7. Open azuredeploy.json
  8. Copy all the contents of azuredeploy.json
  9. Click on “ARM Template”
  10. Delete the contents of the default ARM Template in the web editor
  11. Paste the contents of azuredeploy.json into the web editor
  12. Click on “Add”
  13. Wait for template to create (you may need to click on the “Refresh” button to see it)
  14. Click on Deploy

Part IV: Deploying the template

  1. Set Resource group to the one you created earlier for this Deployment Template
  2. Set Vm SS Name to what you set $scaleSetName to in AzureBootstrap.ps1
  3. Select the Instance Count and Vm Size you want
  4. Set Dns Name Prefix to something arbitrary
  5. Set Admin Username to something arbitrary (This is Windows admin login)
  6. Set Admin Password to something at least 12 characters long, with numbers, and a mix of lower and upper case letters
  7. Copy and paste the URI of your VM image into Source Image Vhd Uri (It looks like “https://storageacct.blob.core.windows.net/system/Microsoft.Compute/Images/destcontainer/template-osDisk.6060b811-6118-4602-abe1-ab06102142fb.vhd”)
  8. Set Bootstrap Script Location to the “Primary blob service endpoint” from earlier with “scripts” appended to the end (i.e: “https://storageacct.blob.core.windows.net/scripts”, with no trailing backslash)
  9. Set Bootstrap Script Storage Account to the same storage account in your Bootstrap Script Location (i.e: “storageacct” in the URI)
  10. Set Bootstrap Script Storage Key to the key for the Storage Account you created to hold the bootstrap script and your Shared Folder (the Storage Account Key from earlier)
  11. Set Image name to something arbitrary, but unique (say, derived from today’s date: “image170607”)
  12. Click on the checkbox at the bottom that says you accept the EULA
  13. Click “Purchase”
  14. Wait for your template to deploy

    Azure Template Deployment Quirks

Azure SQL Server and EFT ARM

(Adapted from Don Mowbray’s DBaas outline)

Part I: Creating an Azure SQL Database

  1. Create a SQL Database Server in Azure DBaaS
  2. Remember the admin login and password you input when creating the SQL Database Server
  3. Wait for SQL Database Server to be provisioned
  4. Create an SQL Database on the SQL database server you created in the first step
  5. Click on the icon for the database just created
  6. Click on the Overview listbox item
  7. Click on the “Set server firewall” tab (to the right, above overview pane)
  8. Click “+ Add client IP”
  9. Fill in Start and End IPs to give your EFT node(s) access to this database
  10. Install SQL Server Management Studio and SQL Server Native Client on your local machine (or local VM)
  11. Retrieve database connection string (ODBC) from SQL Database overview pane
  12. Log into Azure DQL Database in SSMS using information in database connection string

Part II: Setting up the Database

  1. Install EFT on your local machine/vm
  2. Ignore ARM during installation
  3. Navigate to C:Files (x86)Server EnterpriseServer
  4. Open the create_1_tables.sql script in SSMS
  5. Select the database you created using the Available Databases combobox
  6. Click on the Execute button
  7. Open the next script, create_2_primary_keys.sql, in SSMS
  8. Select the database you created using the Available Databases combobox
  9. Click on the Execute button
  10. Open the next script, create_3_foreign_keys.sql, in SSMS
  11. Select the database you created using the Available Databases combobox
  12. Click on the Execute button
  13. Open the next script, create_4_indexes.sql, in SSMS
  14. Select the database you created using the Available Databases combobox
  15. Click on the Execute button
  16. Open the next script, create_5_sprocs.sql, in SSMS
  17. Select the database you created using the Available Databases combobox
  18. Click on the Execute button
  19. Open the next script, create_6_views.sql, in SSMS
  20. Select the database you created using the Available Databases combobox
  21. Click on the Execute button

Part III: Setting up EFT to use Azure SQL Server

  1. Start the EFT Service
  2. Open the EFT Admin UI
  3. Click on the “Server” tab in the left-hand pane of the EFT Admin UI
  4. Click on “LocalHost” (or whatever your server is named)
  5. Click on the “Logs” tab
  6. Click on the “Enable Auditing and Reporting” checkbox so that the checkbox has a check in it
  7. Select SQL Server using the radio button to the right
  8. Enter Database host address using data in connection string
  9. Enter Database name using data in connection string
  10. Select “SQL Server” in the “Authentication” combobox
  11. Enter your username (it’s in the format “username@dbname”)
  12. Enter your password
  13. Click on “Test Connection” button
  14. Click the Apply button
  15. You should see “Connection status: Connected” just above the “Test Connection” button
  16. Create an EFT Site
  17. Perform 2-3 file transfers
  18. Click on the “Report” tab (to the left of the “Status”, “VFS”, and “Server” tabs)
  19. Expand Default Server Group, LocalHost, Reports, and (Globalscape Reports)
  20. Click on “Activity - All File Transfers”
  21. Click on the “Show Report” button to the left
  22. Verify that the report is correct in that it reflects recent EFT activity

Azure Active Directory Domain Services

Adapted in part from Getting Started

  1. Create the Azure AD DC administrators: AAD DC Administrators group
  2. Add appropriate users to the AAD DC Administrators group
  3. Create or select an existing Azure virtual network

Enabling Domain Services

  1. Go to the Classic portal
  2. Navigate to + New -> App Services -> Active Directory -> Directory -> Custom Create
  3. Name your new Directory
  4. Give it a unique domain name
  5. Select country or region
  6. Wait for Directory creation to finish
  7. Click Configure tab
  8. Scroll down to “domain services”
  9. Click on “Yes” for “ENABLE DOMAIN SERVICES FOR THIS DIRECTORY”
  10. Select the virtual network you deployed your Azure VMs to or Create one

If your VMs are on a different Azure Virtual Network from your AADDS

  1. Follow these instructions to connect the two virtual networks

If your VMs are not on an Azure Virtual Network at all

  1. Follow these instructions
  2. Note that, as of 6/22/2017, the VPN client configuration package creates a misconfigured VPN!
  3. Open the VPN connection created by the VPN client configuration package
  4. Copy the URL of the VPN Gateway
  5. Create a new VPN connection, manually
  6. Paste the URL of the created VPN Gateway for the server address
  7. Set “VPN Type” to “automatic”
  8. Set “Type of sign-in info” to “Certificate”
  9. Save your changes
  10. Click “Connect”

Misc AADDS Notes

Using Secure LDAP (LDAPS)

Deployment from Shared Image

  1. For non-production deployments, use US South Central Region to minimize latency.
  2. Create a premium LRS storage account and container to accommodate the VHD image.
  3. Provide (email?) the storage account name, container URL (e.g., https://pswteststorageaccount.blob.core.windows.net/pswtestcontainer) and storage access key to the gold image owner.
  4. Image owner will use AzCopy to copy the gold image into the destination container above.
  5. Follow the deployment steps above to create the Azure Files shared folder, the bootstrap script, and deploy the scale set template.

Scaling with Draining

The following proof of concept was originally a requirement for Azure Enablement, but for the current iteration (June 2017) it is considered a nice-to-have rather a must-have. For project future iterations, it is possible that the metrics and billing infrastructure will drive scale-in and scale-out decisions so we can better manage our costs and associated margins.

  1. Disable autoscaling on the scale set. This is required because we will be explicitly setting scale set capacity, which appears incompatible with autoscaling.
  2. Create an automation account
  3. Create the following runbooks in the automation account.

  4. From the runbook blade, create webhooks for the above runbooks.
  5. From the scale set’s alerts blade, create the desired ScaleIn and ScaleOut alerts and reference the associated webhooks.

  6. From the automation account, go the Assets blade and create an hourly schedule to invoke the webhook associated with PollScaleSetStatus.ps1

  7. Update the AzureBoostrap.ps1 script and supply the webhook associated with DeprovisionVMSSPhase2.ps1 to the AzureConfig.ScaleInPostDrainWebhookUrl under the section that generates AzureConfig.json.
  8. Upload the AzureBootstrap.ps1 script modified with the AzureConfig.ScaleInPostDrainWebhookUrl setting.