New email trigger for Gmail Plugin - Question

Hi!

I’ve been working on triggering workflows with new emails on a particular Gmail inbox using its Plugin ver 6.0.1

Is there any time restriction that I forget to change?

On a daily basis whenever the hour is 16:00 CET and later, the workflow never triggers although every new email gets changed from (is: unread) to (is: read), but the workflow just doesn’t runs.

Thank you!

The fact that the email is changing from read to unread indicates that the trigger did fire, so that’s an interesting nugget.

Have you checked your failed jobs page to see if the workflow failed?

Hey Joey,

That is the tricky part, the job doesn’t appear at all!

Nor closed nor running nor decision nor failed.

I’ve been replicating this for over a week now trying to see if anything was missing.

The workflow runs normally up to 15:59 CET, then at 16:00 CET the state of the mail changes but no job at sight.

If you need more info, let me know (Although I have no clue where to look :slight_smile: )

I’m phoning a friend on this one. Looking at the code in the Gmail plugin makes it seem like timezone shouldn’t be a problem here, and there’s nothing special about 16:00 to my knowledge that would make it fail.

One of the questions that came through…are you using the Gmail trigger in more than one workflow? You don’t have any other workflows that fire off at exactly this time do you?

And this is completely out of left field…you’re not rebooting the orcha at midnight or something weird like that? The reason I ask is if the container is somehow restarted at 16:00, that could cause this too.

Thank you for the insights Joey,

On the first question, we don’t have yet any other workflow being triggered by Gmail.

Will check with our infra engineers if the orchestrator is being rebooted at any time.

After checking all our settings locally on the orchestrator,

There is no indication whatsoever for the reason the workflow doesn’t start even though the mail gets marked as read.

There is no other automation on place for this specific account and Environment - (As we are using a separate Gsuite from production)

I left some emails scheduled over Fri-Sat night.

Screenshot 2020-09-21 at 09.00.12

All the emails reached the inbox and are marked as read.

Screenshot 2020-09-21 at 09.04.51

The last working trigger was at 15:00:58 Friday CET, the next was 06:00:54 Saturday CET.

image

Did you find out if the orcha rebooted in that time frame?

Not at all,

The orchestrator doesn’t reboot at that time.

You can see this workflows that failed, but they appear (Other trigger)

image

Ok, let me talk to the other developers and see what we can come up with. Is your orcha on a different time zone than Gmail by chance?

The orchestrator shows this when we ssh to it…

This Orchestrator is currently configured to point to the following cloud region: [us]
Activation will only occur if the cloud region matches the region where your Rapid7 organization exists.

Maybe this is one reason?

Also we checked out the docker containers and spot that the container for the gmail-trigger-6.0.1 ran for 10 hours, then stopped.

sudo docker ps -a

CONTAINER ID        IMAGE                                COMMAND                  CREATED             STATUS                     PORTS                        NAMES
cb57f31cb72f        rapid7/whois:2.0.3                   "/usr/local/bin/koma…"   10 days ago         Up 10 days                 127.0.0.1:10124->10001/tcp   rapid7_whois_2.0.3_action
347a3e4e8976        rapid7/whois:2.0.2                   "/usr/local/bin/koma…"   10 days ago         Up 10 days                 127.0.0.1:10123->10001/tcp   rapid7_whois_2.0.2_action
8581e9bcb8cc        rapid7/grep:1.0.3                    "/usr/local/bin/icon…"   2 weeks ago         Up 2 weeks                 127.0.0.1:10122->10001/tcp   rapid7_grep_1.0.3_action
b07906758f90        rapid7/type_converter:1.5.2          "/usr/local/bin/icon…"   3 weeks ago         Up 3 weeks                 127.0.0.1:10121->10001/tcp   rapid7_type_converter_1.5.2_action
189d0da7e49e        rapid7/rapid7_vulndb:2.0.3           "/usr/local/bin/koma…"   3 weeks ago         Up 3 weeks                 127.0.0.1:10120->10001/tcp   rapid7_rapid7_vulndb_2.0.3_action
198f2824f96e        rapid7/timers:2.0.4                  "/komand/plugins/bin…"   4 weeks ago         Up 4 weeks                 127.0.0.1:10119->10001/tcp   rapid7_timers_2.0.4_action
d0dc4fcd774e        rapid7/gmail:5.1.4                   "/usr/local/bin/icon…"   4 weeks ago         Exited (137) 4 weeks ago                                rapid7_gmail_5.1.4_trigger_715ca3a5-e554-4843-ab54-0a51dfdc1fef
83951b58cda6        rapid7/gmail:6.0.1                   "/usr/local/bin/icon…"   4 weeks ago         Up 10 hours                                             rapid7_gmail_6.0.1_trigger_1f13fcbb-64f6-4f5a-97c4-4c72c0c1213a
bb4071462783        rapid7/abuseipdb:5.0.3               "/usr/local/bin/koma…"   6 weeks ago         Up 6 weeks                 127.0.0.1:10118->10001/tcp   rapid7_abuseipdb_5.0.3_action
1e191859df46        rapid7/python_3_script:2.0.2         "/usr/local/bin/koma…"   6 weeks ago         Up 6 weeks                 127.0.0.1:10117->10001/tcp   rapid7_python_3_script_2.0.2_action
d50ef68fb375        rapid7/string:1.3.0                  "/usr/local/bin/koma…"   6 weeks ago         Up 6 weeks                 127.0.0.1:10116->10001/tcp   rapid7_string_1.3.0_action
c696e468e1ee        rapid7/gmail:6.0.1                   "/usr/local/bin/icon…"   6 weeks ago         Up 6 weeks                 127.0.0.1:10115->10001/tcp   rapid7_gmail_6.0.1_action
d9d683de9748        rapid7/whois:1.0.3                   "/usr/local/bin/koma…"   7 weeks ago         Up 7 weeks                 127.0.0.1:10114->10001/tcp   rapid7_whois_1.0.3_action
242a7b1958e7        rapid7/extractit:1.1.6               "/komand/plugins/bin…"   7 weeks ago         Up 7 weeks                 127.0.0.1:10113->10001/tcp   rapid7_extractit_1.1.6_action
877bd35a5081        rapid7/ipstack:1.0.0                 "/usr/local/bin/koma…"   7 weeks ago         Up 7 weeks                 127.0.0.1:10112->10001/tcp   rapid7_ipstack_1.0.0_action
8aaed42d0e3a        rapid7/abuseipdb:3.0.1               "/usr/local/bin/koma…"   7 weeks ago         Up 7 weeks                 127.0.0.1:10111->10001/tcp   rapid7_abuseipdb_3.0.1_action
ef6c400be39d        rapid7/string:1.2.1                  "/usr/local/bin/koma…"   8 weeks ago         Up 8 weeks                 127.0.0.1:10110->10001/tcp   rapid7_string_1.2.1_action
7224611abc45        rapid7/extractit:2.0.0               "/komand/plugins/bin…"   8 weeks ago         Up 8 weeks                 127.0.0.1:10109->10001/tcp   rapid7_extractit_2.0.0_action
ae8da3578bfa        rapid7/active_directory_ldap:3.2.9   "/usr/local/bin/koma…"   8 weeks ago         Up 8 weeks                 127.0.0.1:10108->10001/tcp   rapid7_active_directory_ldap_3.2.9_action
495893d6e531        rapid7/gmail:5.1.4                   "/usr/local/bin/icon…"   2 months ago        Exited (1) 6 weeks ago                                  rapid7_gmail_5.1.4_trigger_1f13fcbb-64f6-4f5a-97c4-4c72c0c1213a
911ab0ccdd28        rapid7/gmail:5.1.4                   "/usr/local/bin/icon…"   2 months ago        Up 2 months                127.0.0.1:10107->10001/tcp   rapid7_gmail_5.1.4_action
57a18c31a37b        rapid7/python_3_script:2.0.1         "/usr/local/bin/koma…"   2 months ago        Up 2 months                127.0.0.1:10106->10001/tcp   rapid7_python_3_script_2.0.1_action
ba055fb4aaa1        rapid7/virustotal:4.0.0              "/usr/local/bin/koma…"   2 months ago        Up 2 months                127.0.0.1:10105->10001/tcp   rapid7_virustotal_4.0.0_action
f301785f96f9        rapid7/jira:4.0.2                    "/usr/local/bin/koma…"   7 months ago        Up 7 months                127.0.0.1:10104->10001/tcp   rapid7_jira_4.0.2_action
9882e4e13471        rapid7/active_directory_ldap:3.2.8   "/usr/local/bin/koma…"   7 months ago        Up 7 months                127.0.0.1:10103->10001/tcp   rapid7_active_directory_ldap_3.2.8_action
bfcb118f3b69        rapid7/splunk:3.0.2                  "/usr/local/bin/koma…"   7 months ago        Up 7 months                127.0.0.1:10102->10001/tcp   rapid7_splunk_3.0.2_action
4f761f719cc3        rapid7/jira:4.0.0                    "/usr/local/bin/koma…"   7 months ago        Up 7 months                127.0.0.1:10101->10001/tcp   rapid7_jira_4.0.0_action

We’ve got a ticket in for this now so we can figure out what’s going on here. One other possibility that Joey mentioned was conflicting time zones between the orchestrator and Gmail, resulting in some weird timer issues.