n8n Workflow Automation
Develop workflows, custom nodes, and integrations for the n8n automation platform
- Category: discovered
- Tags: github, community
- Version: 1.0.0
- Author: walid
- Downloads: 0
- Updated: 2025-11-22
Download the latest version (v1.0.0) from this page.
SKILL.md
---
skill_id: n8n
name: n8n Workflow Automation
description: Develop workflows, custom nodes, and integrations for the n8n automation platform
version: 1.0.0
created: 2025-11-13
complexity: 3
---
# n8n Workflow Automation Skill
## Purpose
Provide specialized guidance for developing workflows, custom nodes, and integrations on the n8n automation platform. Enable AI assistants to design workflows, write custom code nodes, build TypeScript-based custom nodes, integrate external services, and implement AI agent patterns.
## When to Use This Skill
Invoke this skill when:
- Designing automation workflows combining multiple services
- Writing JavaScript/Python code within workflow nodes
- Building custom nodes in TypeScript
- Integrating APIs, databases, and cloud services
- Creating AI agent workflows with LangChain
- Troubleshooting workflow execution errors
- Planning self-hosted n8n deployments
- Converting manual processes to automated workflows
Do NOT use this skill for:
- Generic automation advice (use appropriate language/platform skill)
- Cloud platform-specific integrations (combine with cloud provider skill)
- Database design (use database-specialist skill)
- Frontend development (n8n has minimal UI customization)
## Core n8n Concepts
### Platform Architecture
**Runtime Environment:**
- Node.js-based execution engine
- TypeScript (90.7%) and Vue.js frontend
- pnpm monorepo structure
- Self-hosted or cloud deployment options
**Workflow Execution Models:**
1. **Manual trigger** - User-initiated execution
2. **Webhook trigger** - HTTP endpoint activation
3. **Schedule trigger** - Cron-based timing
4. **Event trigger** - External service events (database changes, file uploads)
5. **Error trigger** - Workflow failure handling
**Fair-code License:**
- Apache 2.0 with Commons Clause
- Free for self-hosting and unlimited executions
- Commercial restrictions for SaaS offerings
### Node Types and Categories
**Core Nodes** (Data manipulation):
- **Code** - Execute JavaScript/Python
- **Set** - Assign variable values
- **If** - Conditional branching
- **Switch** - Multi-branch routing
- **Merge** - Combine data streams
- **Split In Batches** - Process large datasets incrementally
- **Loop Over Items** - Iterate through data
**Trigger Nodes** (Workflow initiation):
- **Webhook** - HTTP endpoint
- **Schedule** - Time-based execution
- **Manual Trigger** - User activation
- **Error Trigger** - Catch workflow failures
- **Start** - Default entry point
**Action Nodes** (500+ integrations):
- API connectors (REST, GraphQL, SOAP)
- Database clients (PostgreSQL, MongoDB, MySQL, Redis)
- Cloud services (AWS, GCP, Azure, Cloudflare)
- Communication (Email, Slack, Discord, SMS)
- File operations (FTP, S3, Google Drive, Dropbox)
- Authentication (OAuth2, API keys, JWT)
**AI Nodes** (LangChain integration):
- **AI Agent** - Autonomous decision-making
- **AI Chain** - Sequential LLM operations
- **AI Transform** - Data manipulation with LLMs
- **Vector Store** - Embedding storage and retrieval
- **Document Loaders** - Text extraction from files
### Data Flow and Connections
**Connection Types:**
1. **Main connection** - Primary data flow (solid line)
2. **Error connection** - Failure routing (dashed red line)
**Data Structure:**
```javascript
// Input/output format for all nodes
[
{
json: { /* Your data object */ },
binary: { /* Optional binary data (files, images) */ },
pairedItem: { /* Reference to source item */ }
}
]
```
**Data Access Patterns:**
- **Expression** - `{{ $json.field }}` (current node output)
- **Input reference** - `{{ $('NodeName').item.json.field }}` (specific node)
- **All items** - `{{ $input.all() }}` (entire dataset)
- **First item** - `{{ $input.first() }}` (single item)
- **Item index** - `{{ $itemIndex }}` (current iteration)
### Credentials and Authentication
**Credential Types:**
- **Predefined** - Pre-configured for popular services (OAuth2, API key)
- **Generic** - HTTP authentication (Basic, Digest, Header Auth)
- **Custom** - User-defined credential structures
**Security Practices:**
- Credentials stored encrypted in database
- Environment variable support for sensitive values
- Credential sharing across workflows (optional)
- Rotation: Manual update required
## Workflow Design Methodology
### Planning Phase
**Step 1: Define Requirements**
- Input sources (webhooks, schedules, databases)
- Data transformations needed
- Output destinations (APIs, files, databases)
- Error handling requirements
- Execution frequency and volume
**Step 2: Map Data Flow**
- Identify trigger events
- List transformation steps
- Specify validation rules
- Define branching logic
- Plan error recovery
**Step 3: Select Nodes**
Decision criteria:
- Use **native nodes** when available (optimized, maintained)
- Use **Code node** for custom logic <50 lines
- Build **custom node** for reusable complex logic >100 lines
- Use **HTTP Request node** for APIs without native nodes
- Use **Execute Command node** for system operations (security risk)
### Implementation Phase
**Workflow Structure Pattern:**
```
[Trigger] → [Validation] → [Branch (If/Switch)] → [Processing] → [Error Handler]
↓ ↓
[Path A nodes] [Path B nodes]
↓ ↓
[Merge/Output] [Output]
```
**Modular Design:**
- Extract reusable logic to sub-workflows
- Use Execute Workflow node for modularity
- Limit main workflow to 15-20 nodes (readability)
- Parameterize workflows with input variables
**Error Handling Strategy:**
1. **Error Trigger workflows** - Capture all failures
2. **Try/Catch pattern** - Error output connections on nodes
3. **Retry logic** - Configure per-node retry settings
4. **Validation nodes** - If/Switch for data checks
5. **Notification** - Alert on critical failures (Email, Slack)
### Testing Phase
**Local Testing:**
- Execute with sample data
- Verify each node output (inspect data panel)
- Test error paths with invalid data
- Check credential connections
**Production Validation:**
- Enable workflow, monitor executions
- Review execution history for failures
- Check resource usage (execution time, memory)
- Validate output data quality
## Code Execution in Workflows
### Code Node (JavaScript)
**Available APIs:**
- **Node.js built-ins** - `fs`, `path`, `crypto`, `https`
- **Lodash** - `_.groupBy()`, `_.sortBy()`, etc.
- **Luxon** - DateTime manipulation
- **n8n helpers** - `$input`, `$json`, `$binary`
**Basic Structure:**
```javascript
// Access input items
const items = $input.all();
// Process data
const processedItems = items.map(item => {
const inputData = item.json;
return {
json: {
// Output fields
processed: inputData.field.toUpperCase(),
timestamp: new Date().toISOString()
}
};
});
// Return transformed items
return processedItems;
```
**Data Transformation Patterns:**
*Filtering:*
```javascript
const items = $input.all();
return items.filter(item => item.json.status === 'active');
```
*Aggregation:*
```javascript
const items = $input.all();
const grouped = _.groupBy(items, item => item.json.category);
return [{
json: {
summary: Object.keys(grouped).map(category => ({
category,
count: grouped[category].length
}))
}
}];
```
*API calls (async):*
```javascript
const items = $input.all();
const results = [];
for (const item of items) {
const response = await fetch(`https://api.example.com/data/${item.json.id}`);
const data = await response.json();
results.push({
json: {
original: item.json,
enriched: data
}
});
}
return results;
```
**Error Handling in Code:**
```javascript
const items = $input.all();
return items.map(item => {
try {
// Risky operation
const result = JSON.parse(item.json.data);
return { json: { parsed: result } };
} catch (error) {
return {
json: {
error: error.message,
original: item.json.data
}
};
}
});
```
### Code Node (Python)
**Available Libraries:**
- **Standard library** - `json`, `datetime`, `re`, `requests`
- **NumPy** - Array operations
- **Pandas** - Data analysis (if installed)
**Basic Structure:**
```python
# Access input items
items = _input.all()
# Process data
processed_items = []
for item in items:
input_data = item['json']
processed_items.append({
'json': {
'processed': input_data['field'].upper(),
'timestamp': datetime.now().isoformat()
}
})
# Return transformed items
return processed_items
```
**Complexity Rating: Code Nodes**
- Simple transformations (map/filter): **1**
- API calls with error handling: **2**
- Multi-step async operations: **3**
- Complex algorithms with libraries: **4**
- Performance-critical processing: **5** (consider custom node)
## Custom Node Development
### When to Build Custom Nodes
**Build custom node when:**
- Reusable logic across multiple workflows (>3 workflows)
- Complex authentication requirements
- Performance-critical operations (Code node overhead)
- Community contribution (public npm package)
- Organization-specific integrations
**Use Code node when:**
- One-off transformations
- Rapid prototyping
- Simple API calls (<100 lines)
### Development Styles
[See Code Examples: examples/n8n_custom_node.ts]
**1. Programmatic Style** (Full control)
*Use for:*
- Complex authentication flows
- Advanced parameter validation
- Custom UI components
- Polling operations with state management
[See: `CustomNode` class in examples/n8n_custom_node.ts]
**2. Declarative Style** (Simplified)
*Use for:*
- Standard CRUD operations
- RESTful API wrappers
- Simple integrations without complex logic
[See: `operations` and `router` exports in examples/n8n_custom_node.ts]
**Additional Examples:**
- Credential configuration: `customApiCredentials` in examples/n8n_custom_node.ts
- Credential validation: `validateCredentials()` in examples/n8n_custom_node.ts
- Polling trigger: `PollingTrigger` class in examples/n8n_custom_node.ts
### Development Workflow
**Step 1: Initialize Node**
```bash
# Create from template
npm create @n8n/node my-custom-node
# Directory structure created:
# ├── nodes/
# │ └── MyCustomNode/
# │ └── MyCustomNode.node.ts
# ├── credentials/
# │ └── MyCustomNodeApi.credentials.ts
# └── package.json
```
**Step 2: Implement Logic**
- Define node properties (parameters, credentials)
- Implement execute method
- Add error handling
- Write unit tests (optional)
**Step 3: Build and Test**
```bash
# Build TypeScript
npm run build
# Link locally for testing
npm link
# In n8n development environment
cd ~/.n8n/nodes
npm link my-custom-node
# Restart n8n to load node
n8n start
```
**Step 4: Publish**
```bash
# Community node (npm package)
npm publish
# Install in n8n
Settings → Community Nodes → Install → Enter package name
```
**Complexity Rating: Custom Nodes**
- Declarative CRUD wrapper: **2**
- Programmatic with authentication: **3**
- Complex state management: **4**
- Advanced polling/webhooks: **5**
## Integration Patterns
### API Integration Strategy
**Decision Tree:**
```
Has native node? ──Yes──> Use native node
│
No
├──> Simple REST API? ──Yes──> HTTP Request node
├──> Complex auth (OAuth2)? ──Yes──> Build custom node
├──> Reusable across workflows? ──Yes──> Build custom node
└──> One-off integration? ──Yes──> Code node with fetch()
```
### HTTP Request Node Patterns
**GET with query parameters:**
```
URL: https://api.example.com/users
Method: GET
Query Parameters:
- status: active
- limit: 100
Authentication: Header Auth
- Name: Authorization
- Value: Bearer {{$credentials.apiKey}}
```
**POST with JSON body:**
```
URL: https://api.example.com/users
Method: POST
Body Content Type: JSON
Body:
{
"name": "={{ $json.name }}",
"email": "={{ $json.email }}"
}
```
**Pagination handling (Code node):**
```javascript
let allResults = [];
let page = 1;
let hasMore = true;
while (hasMore) {
const response = await this.helpers.request({
method: 'GET',
url: `https://api.example.com/data?page=${page}`,
json: true,
});
allResults = allResults.concat(response.results);
hasMore = response.hasNext;
page++;
}
return allResults.map(item => ({ json: item }));
```
### Webhook Patterns
**Receiving webhooks:**
1. Create webhook trigger node
2. Configure HTTP method (POST/GET)
3. Set authentication (None/Header Auth/Basic Auth)
4. Get webhook URL from node
5. Register URL with external service
**Responding to webhooks:**
```javascript
// In Code node after webhook trigger
const webhookData = $input.first().json;
// Process data
const result = processData(webhookData);
// Return response (synchronous webhook)
return [{
json: {
status: 'success',
data: result
}
}];
```
**Webhook URL structure:**
```
Production: https://your-domain.com/webhook/workflow-id
Test: https://your-domain.com/webhook-test/workflow-id
```
### Database Integration
**Common patterns:**
*Query with parameters:*
```sql
-- PostgreSQL node
SELECT * FROM users
WHERE created_at > $1
AND status = $2
ORDER BY created_at DESC
-- Parameters from previous node
Parameters: ['{{ $json.startDate }}', 'active']
```
*Batch insert:*
```javascript
// Code node preparing data for database
const items = $input.all();
const values = items.map(item => ({
name: item.json.name,
email: item.json.email,
created_at: new Date().toISOString()
}));
return [{ json: { values } }];
// Next node: PostgreSQL
// INSERT INTO users (name, email, created_at)
// VALUES {{ $json.values }}
```
### File Operations
**Upload to S3:**
```
Workflow: File Trigger → S3 Upload
- File Trigger: Monitor directory for new files
- S3 node:
- Operation: Upload
- Bucket: my-bucket
- File Name: {{ $json.fileName }}
- Binary Data: true (from file trigger)
```
**Download and process:**
```
HTTP Request (download) → Code (process) → Google Drive (upload)
- HTTP Request: Binary response enabled
- Code: Process $binary.data
- Google Drive: Upload with binary data
```
## AI Agent Workflows
### LangChain Integration
**AI Agent Node Configuration:**
- **Agent type**: OpenAI Functions, ReAct, Conversational
- **LLM**: OpenAI, Anthropic, Hugging Face, Ollama (local)
- **Memory**: Buffer, Buffer Window, Summary
- **Tools**: Calculator, Webhook, Database query, Custom API calls
**Basic Agent Pattern:**
```
Manual Trigger → AI Agent → Output
- AI Agent:
- Prompt: "You are a helpful assistant that {{$json.task}}"
- Tools: [Calculator, HTTP Request]
- Memory: Conversation Buffer Window
```
### Gatekeeper Pattern (Supervised AI)
**Use case:** Human approval before agent actions
```
Webhook → AI Agent → If (requires approval) → Send Email → Wait for Webhook → Execute Action
↓ (auto-approve)
Execute Action
```
**Implementation:**
1. AI Agent generates action plan
2. If node checks confidence score
3. Low confidence → Email approval request
4. Wait for webhook (approve/reject)
5. Execute or abort based on response
### Iterative Agent Pattern
**Use case:** Multi-step problem solving with state
```
Loop Start → AI Agent → Tool Execution → State Update → Loop End (condition check)
↑______________________________________________________________|
```
**State management:**
```javascript
// Code node - Initialize state
return [{
json: {
task: 'Research topic',
iteration: 0,
maxIterations: 5,
context: [],
completed: false
}
}];
// Code node - Update state
const state = $json;
state.iteration++;
state.context.push($('AI Agent').item.json.response);
state.completed = state.iteration >= state.maxIterations || checkGoalMet(state);
return [{ json: state }];
```
### RAG (Retrieval Augmented Generation) Pattern
```
Query Input → Vector Store Search → Format Context → LLM → Response Output
```
**Vector Store setup:**
1. Document Loader node → Split text into chunks
2. Embeddings node → Generate vectors (OpenAI, Cohere)
3. Vector Store node → Store in Pinecone/Qdrant/Supabase
4. Query: Retrieve relevant chunks → Inject into LLM prompt
**Complexity Rating: AI Workflows**
- Simple LLM call: **1**
- Agent with tools: **3**
- Gatekeeper pattern: **4**
- Multi-agent orchestration: **5**
## Deployment and Hosting
### Self-Hosting Options
[See Code Examples: examples/n8n_deployment.yaml]
**Docker (Recommended):**
- Docker Compose with PostgreSQL
- Queue mode configuration for scaling
- Resource requirements by volume
[See: docker-compose configurations in examples/n8n_deployment.yaml]
**npm (Development):**
```bash
npm install n8n -g
n8n start
# Access: http://localhost:5678
```
**Environment Configuration:**
[See: Complete environment variable reference in examples/n8n_deployment.yaml]
Essential variables:
- `N8N_HOST` - Public URL for webhooks
- `WEBHOOK_URL` - Webhook endpoint base
- `N8N_ENCRYPTION_KEY` - Credential encryption (must persist)
- `DB_TYPE` - Database (SQLite/PostgreSQL/MySQL/MariaDB)
- `EXECUTIONS_DATA_SAVE_ON_ERROR` - Error logging
- `EXECUTIONS_DATA_SAVE_ON_SUCCESS` - Success logging
Performance tuning variables documented in examples/n8n_deployment.yaml
### Scaling Considerations
**Queue Mode (High volume):**
```bash
# Separate main and worker processes
# Main process (UI + queue management)
N8N_QUEUE_MODE=main n8n start
# Worker processes (execution only)
N8N_QUEUE_MODE=worker n8n worker
```
**Database:**
- SQLite: Development/low volume (<1000 executions/day)
- PostgreSQL: Production (recommended)
- MySQL/MariaDB: Alternative for existing infrastructure
**Resource Requirements:**
| Workflow Volume | CPU | RAM | Database |
|----------------|-----|-----|----------|
| <100 exec/day | 1 core | 512MB | SQLite |
| 100-1000/day | 2 cores | 2GB | PostgreSQL |
| 1000-10000/day | 4 cores | 4GB | PostgreSQL |
| >10000/day | 8+ cores | 8GB+ | PostgreSQL + Queue mode |
**Monitoring:**
- Enable execution logs (EXECUTIONS_DATA_SAVE_*)
- Set up error workflows (Error Trigger node)
- Monitor database size (execution history cleanup)
- Track webhook response times
## Best Practices
### Workflow Design
**1. Modularity:**
- Extract reusable logic to Execute Workflow nodes
- Limit workflows to single responsibility
- Use sub-workflows for common operations (validation, formatting)
**2. Error Resilience:**
- Add error outputs to critical nodes
- Implement retry logic (node settings)
- Create Error Trigger workflows for alerts
- Log errors to external systems (Sentry, CloudWatch)
**3. Performance:**
- Use Split In Batches for large datasets (>1000 items)
- Minimize HTTP requests in loops (batch API calls)
- Disable execution logging for high-frequency workflows
- Cache expensive operations in variables
**4. Security:**
- Store secrets in credentials (not hardcoded)
- Use environment variables for configuration
- Enable webhook authentication
- Restrict Execute Command node usage (or disable globally)
- Review code nodes for injection vulnerabilities
**5. Maintainability:**
- Add notes to complex workflows (Sticky Note node)
- Use consistent naming (verb + noun: "Fetch Users", "Transform Data")
- Document workflow purpose in workflow settings
- Version control workflows (export JSON, commit to Git)
### Code Quality in Nodes
**1. Data validation:**
```javascript
// Always validate input structure
const items = $input.all();
for (const item of items) {
if (!item.json.email || !item.json.name) {
throw new Error(`Invalid input: missing required fields at item ${item.json.id}`);
}
}
```
**2. Error context:**
```javascript
// Provide debugging information
try {
const result = await apiCall(item.json.id);
} catch (error) {
throw new Error(`API call failed for ID ${item.json.id}: ${error.message}`);
}
```
**3. Idempotency:**
```javas