When working with potentially malicious URLs, the first problem you usually face is “where does this URL actually take me?” Attackers routinely hide their malicious pages behind url shorteners, redirectors, etc. to try and get past url filters and to make the link look as “normal” as possible. This technique also works on many url scanning solutions since they scan a page but generally don’t follow redirects within the page (for example, onload functions that redirect elsewhere). They get a blank page, and return a “clean” result.
The easiest way to discover the ultimate destination of a page is to send it through a web browser inside a sandbox and see where it goes. While powerful, this approach is also complicated, time consuming, and usually fairly expensive to implement (both in time and money - sandboxes aren’t cheap!).
Fortunately, the urlscan.io service exists and performs sandbox-like analysis on URLs providing the ultimate URL the link takes you to and a screenshot of the page, in addition to a huge amount of additional data. Coupled with decoding rewritten URLs with the ATP and Proofpoint plugins (and others), the urlscan.io plugin can give you the ultimate URL which can then be used for more detailed analysis through tools like VirusTotal.
The workflow generally goes like this:
- check to see if the url is rewritten with ATP or a similar service
- submit the url to urlscan.io
- get the results from urlscan.io - the ultimate URL is returned in the ‘page’ attribute at
{{["urlscan results step name"].[page].[url]}} - submit this url to VirusTotal for analysis
While not perfect, this little trick can circumvent many techniques used to hid the ultimate page of a URL and provide your URL analysis workflows with superior results!