Update Quest RapidRecovery Agents silently

I’m in the process of upgrading AppAssure 5.4.3 agents to the latest version of Rapid Recovery, 6.1.3. This is done following the Core upgrade.

So far my progress has been pretty seamless. In testing I confirmed that I could push the install with a “No Reboot” force, and the agent would continue to operate normally until the next system reboot (which will occur as part of monthly Windows Updates).

This script is in no way ‘complete’, as in it could be made much better. Additional logging on success or failure, parallel processing, etc could all be done -however for simple purposes it worked quite well.

# begin a transcript to log the output to file
Start-Transcript C:\Temp\AgentUpgradeLog.txt
# Define the list of systems to update
# Gather list of older version from Rapid Recovery Licensing Portal
$systems = Get-Content -Path C:\Temp\AgentServers.txt
 
foreach($system in $systems) {
#Write-Host "Testing connectivity to $system"
$rtn = Test-Connection -CN $system -Count 1 -BufferSize 16 -Quiet
 
IF($rtn -match 'True'){
 Write-Host "ALIVE - $System"
 copy C:\Temp\Agent-X64-6.1.3.100.exe \\$system\c$\temp -confirm:$false
 $executable = "C:\temp\Agent-X64-6.1.3.100.exe"
 $session = New-PSSession -ComputerName $system 
 # Key here is the /silent and reboot=never switches
Invoke-Command -Session $session -ScriptBlock {Start-Process $using:executable -ArgumentList "/silent reboot=never" -Wait}
 remove-pssession $session
 Write-Host "$System - upgrade steps completed"
 sleep 5
# Found that we had agent services not starting following the upgrade
 get-service -name "RapidRecoveryAgent" -computername $system | Start-Service
 }
 
else
 {
 Write-Host "DOWN - $System"
 }
 
}
 
Stop-Transcript

 

Update drive label with no letter

In the process of re-organizing naming structure of a set of Cluster Shared Volumes, I had a need to rename the volume upon an iSCSI disk that was presented to the cluster.

Originally I sourced this article which requires a drive letter associated with the volume in order to find the proper object.

Following that, I came across this article which had the syntax I needed in the last post.

Here’s the script I used successfully:

$drive = gwmi win32_volume -filter "Label = 'Old Name'"
$drive.Label = "New Name"
$drive.put()

Intel Network Connections (ProSet) update or removal

Short-version: be very careful removing the Intel Network Connections software!

I’m working on a task to update NIC drivers and firmware on a Dell PowerEdge R630. It was decided during this process to remove the Intel ProSet software along with it’s generated sub-interfaces for VLAN purposes, and instead utilize switch-port configuration.

I went to uninstall Intel(R) Network Connections from appwiz.cpl, which contains ProSet. On this server, its version was 19.3.0.0. However, I quickly received an error stating that I needed to remove any Teams and VLANs. No problem, I was going to do that anyways so I went into the advanced properties of my NICs and removed the VLAN sub-interface. I wasn’t using any Intel teaming, but rather LFBO within Windows Server.

When I tried again, it still gave me the same error. I decided to try “Modify” on the uninstaller instead, and remove ProSET only.

This was successful, but then when I tried to install the latest version, it said it couldn’t update with this error:
Product: Intel(R) Network Connections — The installed version of Intel(R) Network Connections is not supported for upgrades. You must uninstall it before installing this version.

So I tried the full uninstall again, and this time it worked! Great, I’m thinking “Now I can proceed with the latest version”.

Except it removed ALL NIC config, including IP settings and advanced properties like Jumbo Frames and RSS buffer sizes.

Thankfully I had good documentation of the previous setup, and that should be your biggest takeaway if you’re reading this: always consider “what’s the worst that could happen” and then plan to mitigate that thing.

For future hosts, I left the old version of the software installed and manually updated drivers from the .INF files. Interesting to note, if you already have a higher version installed like Version 22.7, this behavior isn’t seen – you can install an update on top of it without having to uninstall.

 

 

 

Rapid Recovery – Force Snapshot selection empty

Using Rapid Recovery 6.1.3, on a Core protecting agents with a very low SLA, I wanted to take a manual snapshot.

When I went to do so, I found the selection list empty.


This is because none of the volumes on the agent were scheduled for protection.

I temporarily added a schedule, forced a snapshot, and then removed the schedule.

I think this is terrible design – there are legitimate reasons to avoid a schedule for some resources, and requiring configuration changes in order to instigate a snapshot introduces risk to un-managed change in an environment.

It goes further though; as this KB indicates, when you have volumes all scheduled and want just a single volume snapshot, you have to remove the schedule (better hope you took note of the settings) and then re-add it afterwards.

There are many little things like this I’ve seen in the AppAssure/Rapid Recovery product since being introduced to it that are quickly souring my opinion, especially coming from a Commvault environment which is extremely flexible and powerful.

Quest Data Protection Portal port requirements

Quest has a new Dashboard portal (Data Protection Portal – DPP) for their Rapid Recovery product, found here: https://dataprotection.quest.com

This gives a very nice overview of your environment including Cores, Machines, and Repositories, along with some level of control. Having recently been exposed to the AppAssure/Rapid Recovery product, this is a very welcome addition because the existing tools like the Central Console are very lacking in central management capabilities. I’ll do a write-up of the capabilities of this Dashboard at a later date.

The DPP requires a plugin to be installed on each Core, which will then communicate with Quest. However, I’ve got Cores in an environment with restricted Internet connectivity. Previously firewall rules were enabled to provide connectivity to the Quest License Portal over HTTP/S, but these are not sufficient for access to the new DPP.

When you try to install the plugin on a Core with limited connectivity, you’ll receive this error before the install wizard appears:

Error for Portal Plugin

Unfortunately, Quest has very little detail on what ports are required, feedback which I’ve already passed along. Their Default Ports page lists connectivity requirements for various functions but not the DPP. In addition, it doesn’t list the destination names or IP in order to isolate outbound HTTP/S even further.

To begin resolving this, I installed WireShark on my Core, began a session, and then attempted to install the Plugin again. Right before the error appeared, I saw DNS request for “s3.amazonaws.com”, and then repeated attempts to an IP over HTTP. I knew I was on the right track because when you download the plugin from the DPP, the link directs to S3:

Plugin download from Quest

I added the FQDN “s3.amazonaws.com” as an address object permitted for outbound HTTP/s access, and attempted to install the plugin again. This time I received a successful prompt to begin the install wizard.

When I run WireShark now, I can see the DNS query, requests for a manifest, and then further on a switch to HTTPs communication (rather than port 80 that it starts with):

Wireshark trace of successful packets

I’d really love to restrict this rule down further than the FQDN of S3, so I’ve opened a case with Quest and will update this post if I get a better option.

Now that the install is done, I went looking at the DPP to ensure my Core was communicating, and found that it was not.

Back to WireShark, I started another session and saw a different set of traffic:

Wireshark trace of DPP communication

A DNS request for “api.licenseportal.com” is completed, and then HTTPs communication that is pretty clearly traffic to the DPP. If one does a reverse DNS lookup on “dataprotection.quest.com” you get an Azure-bound IP address (related to the CNAME resolution in the screenshot above), and if you hit the resolved IP in a browser, you get a certificate warning with a wildcard cert for *.licenseportal.com which is a known Quest domain name.

I added this as an additional FQDN object, and now the Core is displaying data within the DPP. I have now removed the S3.amazonaws.com permission as it is only apparently necessary for the installation.