Building a workflow from scratch in InsightConnect can be complicated. Where do I start, how do I organize it, and how do I make it maintainable? A little up-front thinking and picking the right strategy for your workflow can make the difference between a tangled web of logic and an easily understood (and extensible) workflow.
Before we dive into the specific organizational strategies, I wanted to start with a couple general principles for InsightConnect workflows:
- Use wordy outcome-focused names for steps - names that describe the intended purpose of a step makes the step understandable and identifiable in later steps
- Always keep the end goal in mind - when making a decision about how to do something, keep in mind how the output is going to be used later
- Build for extensibility - try not to back yourself into a corner
These strategies are not meant to be rigid. In fact, many workflow authors combine these strategies when drafting a single workflow.
Strategy 1 - linear progression - basic
Ideal Workflow: Small with minimal potential outcomes
This is the most straightforward (linear, get it?) strategy which is most applicable to small workflows that contain no, or trivial, decision points. You can think of this strategy as a rail - start here, then do this, then do this, etc. until you get to your desired result. While this strategy is the simplest to get started, it also has the least intentional structure which can limit the options you have without reworking pieces of the workflow to accommodate them. Finally, it is driven by the trigger - the overarching question is “what can I do with this data?”
Pros:
- Quick development with low design overhead
- Linear testing - can validate workflow as you work out the logic
Cons:
- End states not determined as you go
- No overall shape can lead to placing steps in illogical places leading to “spaghetti” workflows where it’s hard to find a step
- Potentially difficult to maintain and modify since all parts of the workflow are equal
Example Use Cases
- Alerting on an event
- Processing a vulnerability exception
Strategy 2 - reverse linear - intermediate
Ideal Workflow: Small with single result
This strategy is the inverse of the linear progression - instead of starting with the trigger, you start with the result. The overarching question is “How can I accomplish this outcome?” Starting with the goal keeps you focused on the outcome and ensures that your workflow generates the required data.
You start with the trigger, because you always do, but you can actually start with a temporary API or Slack trigger and swap it out later. You then add your final step - for example “isolate an asset”. Finally, you fill in the steps between the trigger and the outcome gathering the data the outcome needs and adding decision points where the data could be unavailable.
Pros:
- Focused on the outcome
- Minimizes unnecessary steps and eliminates “surprise” reformatting when your desired outcome needs different data than your flow has generated
- Overall shape of workflow is evident early on
Cons:
- High drafting overhead since you must edit steps to add data from steps higher up the workflow
- Potential for workflow logic to be unclear
- Potentially difficult to maintain and modify since all parts of the workflow are equal
Example Use Cases
- Disabling an asset or user account
- Scanning for a vulnerability
Strategy 3 - scaffold - intermediate/advanced
Ideal Workflow: Medium to Large with complex business logic and/or multiple outcomes
This strategy focuses on the shape of the workflow instead of the specific trigger or result. The overarching question is “what should this process look like?” As the name suggests, you start by scaffolding out the logic of your workflow. “What decisions need to be made?” “What are the potential results of the workflow?” You design this logical scaffold first and then fill in the steps required to make these decisions and take action as a result of the decisions.
Pros:
- Facilitates comprehensive decision making
- Allows for easy swapping of data sources used to make decisions
- Business process can be abstracted from implementation - decision points represent the logic from the very beginning
Cons:
- Overarching understanding of the process is required up-front
- Decisions are put in with temporary conditions necessitating validating they’ve all been updated prior to activating the workflow
- Modifications can be difficult since the logic may be quite complex
Example Use Cases
- Certifying an asset as “clean” prior to inclusion in production environment
- Processing a vulnerability exception request
- Onboarding/Offboarding a user
Strategy 4 - section-based - intermediate
Ideal Workflow: Medium to Large with considerable breadth but limited outcomes
This strategy focuses on the types of actions and data the workflow does and needs. The overarching question is “what types of things do I need to do in this workflow?” It is closely related to a scaffold, but is focused on the class of steps in each section instead of on the logic of the workflow.
In this strategy, you break your process up into sections. A common strategy is to identify each section with an artifact step. Let’s take a phishing response workflow as an example. This workflow is triggered by a report of a potential phishing email. Your workflow has 4 sections - report validation, gathering data about the email (enrichment), making a determination about the email, and remediating the email. Once you’ve identified the sections, you fill each section with the actions required to achieve that section’s result.
Going back to our example:
- Validation: confirm that the email was properly reported and has sufficient data included to perform an analysis (aka the user attached the original message to the report)
- Enrichment: extract the data from the email and gather data about URLs from reputation and virus scanning services, running files through virus scanners and sandboxes, gathering data about the e-mail path and headers, etc.
- Determination: make automated decisions based on enrichment data and compile enrichment data into reports to drive human decision points
- Remediation: notify the reports of the analysis outcome, purge the email from the email infrastructure, block detected threats, etc.
Pros:
- Easy to enhance process with additional services and actions
- Maintenance and troubleshooting is simplified since each section is identified
- Placement of steps within the workflow is easily identified (URL analysis sources go in the “gather url information” loop)
- “Anchor points” can be established between sections (such as a join step that abstracts data from one section for the next)
Cons:
- Overall requirements must be identified up-front
- Adding a new section can necessitate adding steps to previous sections (unless you break the organizational strategy)
- Logic can become unclear when handling optional components within a section
Example Use Cases
- Phishing Analysis
- Responding to a security event