A few days ago we installed the Insight agent v2.7.22 on Ubuntu 20.4 expecting it to auto-update to the latest version (v4.0.0.1), but it has not updated yet.
How quickly are agents supposed to auto-update, provided they have a working connection to the Insight Platform?
Is the agent unable to autoupdate if it is more than one major version behind?
Good question. I can answer from my own experience. In the past I deployed agents that were on v3 and it updated the agents automatically within a day or so of v4 being available. I have not tried this with a v2 agent though.
The official documentation from Rapid7 suggests agents above v2 should support automatic updates. Below are some potentially helpful resources. Worst case, you can download v3 through the following link and deploy it to another Ubuntu server. Give it a day or so to see if an auto-update is performed: https://docs.rapid7.com/insight-agent/insight-agent-version-history/
This might help you pinpoint whether the issue is with v2 or the environment. Additional helpful resources are below.
Thanks for chiming in, @Cyb3r
Previously, we have deployed v2.7.22 to Windows Servers and clients, and seen the agent update to the latest supported version within hours to a day, but this is our first install of v2.7.22 after v4.0.0 was released (and possibly our first install of v2.7.22 on Linux).
I also noticed just now that v3.1.2.38 on Windows Server 2012R2 has not updated to the latest agent v3.2.5 supported for R2.
According to documentation, v2.7.22 should able to do auto-updates, but apparently there are caveats regarding agent versions and platform OSs.
I have also noticed the same behavior on Windows Server 2012 R2. However, Iād need to take a look at one of our agents to see what a successful autoupdate event looks like in the logs.
Have you tried looking at the agent.log on the Ubuntu system to see if it is throwing any errors?
/opt/rapid7/ir_agent/components/insight_agent/common/agent.log
Iāll have a look at the Ubuntu log next week, thanks for pointing it out 
I donāt know where to look for updates in the logs, but in the Insight Platform - Agents, each host has the agent version listed with a āLast updatedā timestamp. For all of our 2012 servers, it is āN/Aā.
Insight Agent requirements - operating system support | Insight Agent Documentation says
How OS supportability affects agent functionality
The Insight Agent software receives regular updates (including new features, improvements, and defect fixes) designed to maintain agent performance for all supported OS versions. Running the agent on a supported version ensures that the agent software continues to receive these updates. Rapid7ās Customer Support team can also assist with any questions and troubleshoot any issues that arise with agents installed on supported OS versions.
Reading between the lines, UNsupported OS versions may not get ANY updates, and they may not be automatically updated to the latest available agent for that OS. However, Windows Server 2012 is listed as EOL for Insight Agent Support on 2023-10-31, so the agent should have auto-updated, at least up until that date.
The same holds for Ubuntu 20.04, having EOL for Insight Agent Support on 2030-04-02.
Thanks for pasting this here. You are correct, Windows Server 2012 is under support until Oct 31, 2023. However, Windows Server 2012 R2 went EOL on Apr 10, 2023. The max supported versionāaccording to the documentationāis 3.2.5 which explains why your systems may not have upgraded from v3.1.2.38 to v3.2.5. Double check your assets page to see whether your servers are R2 or non-R2, they are both under different support schedules according to the article.
Correct, thatās what it says - well spotted! Iām very surprised that Rapid7 ended agent support for the āsuccessorā 2012 R2 before itās predecessor 2012, and well before the OS went out of (free) support from Microsoft. It would be interesting to know the reasoning behind that decision.
Our 2012 R2 servers have Rapid7 agent v3.1.2.38 while our 2012 servers have v4.0.0.1.
Anyways, it does not explain why Ubuntu 20.04 (under OS and agent support) did not autoupdate.
I am curious to hear their response as well. If I have time tomorrow, I will try to deploy a v3 agent to one of our Ubuntu 20.04 systems and see how it long it takes to updateāand if there are any logs in agent.log that indicate the prerequisite checks.
Okay, I deployed v2.7.22.6 to one of my test Ubuntu 20.04 machines. Letās give it a few hours or until end of day for results.
It has been 24 hours and my system on v2.7.22.6 has not auto updated. I uninstalled the v2 agent and deployed v3.3.6.32 a few hours ago. I looked at the config.json file for both versions and following values were the same across both,
āenable_updatesā: true,
āp2p_update_enabledā: true,
Upon checking the logs, it appears my system successfully updated to v4.0.0.1. This further proves that despite the config.json stating updates are set trueāI presume this means auto-updatingāit did not update from v2 to v4. Perhaps, v2 is not capable of auto-updating due to it being an older release? My system updated without any issues once I redeployed v3 with no changes made to my network in-between deployments.
Additional data from agent.logāTimezone in UTC -500.
2023-11-22 14:36:28,081 [INFO] [agent.agent]: Agent Semantic Version: 3.3.6.32
2023-11-22 15:08:11,473 [INFO] [agent.agent_socket.SMS.139706545456688]: SocketTracker host(us3.endpoint.ingress.rapid7.com:443) jailed(False) status(NSocketStatus.ACTIVE) status updated to ACTIVE.
2023-11-22 15:08:15,467 [INFO] [root]: Removing event listener - <bound method AgentSocket._update_agent_socket_state of <AgentSocket @ 0x7f0aa6098090 Sent: 350391 Recv: 14087>>
2023-11-22 15:13:01,355 [INFO] [agent.agent]: Agent Semantic Version: 4.0.0.1
Based on the data in the logs it took approximately 45m to update and it utilized the following host for its communications us3.endpoint.ingress.rapid7.com:443.
I would recommend uninstalling the v2 agent. Redeploying a v3 agent and then waiting a few hours to see if it updates without making any modifications to UFW or other firewall technologies. If it updates successfully on v3 then that proves something is not functioning correctly on v2. Similarly, you can package the latest agent installer v4.0.0.1 and use that moving forward which seems to work on auto-update. If it still does not update on v3 then make sure your systems can communicate to us3.endpoint.ingress.rapid7.com:443 which appears to be a precursor to the successful agent update events.
Hopefully this helps.
I can shed some light on this!
The reason for this specific case, with Windows Server 2012 R2, is that a native library used by the Agent (an XML parser) stopped supporting the OS in April of this year and led to us stopping our support for the OS as well.
The release notes in April covers this, but not in the greatest detail - and I donāt have any further insight on the specific here, Iām afraid: Insight Agent Release Notes
Another thing to note is that the primary driver in our OS support list is the technologies used by the Agent. Since itās built using Python and Golang, we follow the support of these languages on the OS level as well.
Golang specifically only supports Windows operating systems as long as theyāre supported by Microsoft, and this doesnāt take things like ESU into account.
For updating from version 2.x on Unix, Iāve seen the same in a few environments lately and Iām currently running some lab tests on a few different distributions. I donāt have a clear answer yet, but will try to update this thread with what I find.
@Cyb3r - Thank you for testing it thoroughly, and confirming that agent v2.7.22.6 will not autoupdate to the current version (v4.0.0.1), whereas a v3.x (v3.3.6.32) will.
We reinstalled using v4.0.0.1, and we expect autoupdate to work again going forward.
@kneser - Thank you for sharing your knowledge, it is much appreciated
Windows Server 2012 R2 went EOS from Microsoft together with Windows Server 2012 on 2023-10-10, so discontinuing active agent support on on 2023-04-03 for an OS that was still supported by Microsoft at the time (not ESU) is a very disturbing move.
It is also surprising that the agent autoupdate function cannot update any autoupdate capable agent to the latest available agent supported for the platform. Apparently, autoupdate fails to update if the latest and greatest agent version is not supported on the current OS platform OR if updating will skip over a major version, i.e. v2.x to 4.y. These caveats are important and should be visibly documented.
Hey! A few quick updates here, sorry itās a little late - busy couple of weeks
Windows Server 2012 R2 EOL:
Unfortunately, I donāt have any further context around the EOL side of things. Ultimately itās a decision made by the Product Managers for the Agent, and it mostly revolves around prioritising support for currently in-date operating systems.
At a guess, and only a guess (this is not authoritative information), the work required to keep supporting Server 2012 R2 might have taken too much resources or perhaps could not have been completed prior to the OS ending general support from Microsoftās side.
Generally Iād suggest getting in touch with your Customer Success Manager if you want to dig deeper into this decision. I know a few customers have opted to go via this route, but I donāt know the outcomes there.
Agent updates on version 2.7.x:
As for the autoupdate issues with older Agent versions, Iāve got an answer on this. Operating System returns were introduces in bootstrap version 2.3.0.3, which shipped with Agent version 3.0.8. Prior to these versions, bootstrap doesnāt return the OS of the asset the Agent is running on.
Agent updates are handled via the bootstrap, in which our Platform pushes Agent updates in accordance with organisation settings and such. As part of this, we have a service that prevents updates to certain operating systems that are no longer supported on newer versions. This service requires there to be an OS return; if there is a null value, it wonāt push any updates to that bootstrap.
So, given that youāre running Agents below version 3.0.8 in which the OS return was introduced, the service would not push any updates out to them on account of no OS return.
From chatting to dev about this, thereās no great way of adjusting the service to allow for updates without causing further problems as we have no way of knowing which operating systems they run on from a bootstrap perspective.
As an example of why removing the rule is not ideal, if we did so then any Agents running on Windows Server 2008 or Windows 7 SPO1, with a last-supported version of 2.7.22, would update to 3.0.8 which would break the installation and the Agent would not function anymore.
Our approach currently is to prevent these situations, and we currently recommend customers manually upgrade Agents below 3.0.8 that are supported up to or above this version. Itās an ongoing discussion to see if we can find a way around the limitations here without unintentionally breaking things, but Iād go with manual updates as I donāt expect us to make any changes here in the foreseeable future given itās near the end of the year.
And for reference, the last supported versions for out-of-support operating systems are listed here:
Great information, thank you!