Skip to content

Commit 8acf8d3

Browse files
committed
feat: Implement connection rules and rate limiting
- Add retry, filter, transform, delay, and deduplicate rules - Support destination rate limiting (per second/minute/hour) - Add comprehensive acceptance tests - Update design documentation
1 parent 2ea9e9e commit 8acf8d3

File tree

4 files changed

+981
-4
lines changed

4 files changed

+981
-4
lines changed
Lines changed: 219 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,219 @@
1+
# Connection Rule Configuration CLI Design
2+
3+
This document outlines the design for command-line flags to configure ordered rules for Hookdeck connections.
4+
5+
## 1. Guiding Principles
6+
7+
1. **Clarity and Predictability**: The mapping from flags to API rules should be intuitive. Users should be able to predict the outcome based on the flags they provide.
8+
2. **Order Matters**: The CLI must respect the order of flags to determine the execution order of the rules. The first rule-related flag encountered sets the position of that rule in the execution chain.
9+
3. **Consistency**: Flag names should follow the existing conventions outlined in `AGENTS.md`, using a `rule-<type>-<property>` pattern.
10+
4. **Atomicity for Complex Rules**: For rules with multiple properties (like `filter`), the first flag of that rule type determines its order. Subsequent flags for the same rule type will amend the existing rule object rather than creating a new one.
11+
5. **Escape Hatch**: A JSON-based fallback mechanism (`--rules` and `--rules-file`) will be provided for complex configurations that are difficult to express with flags.
12+
13+
14+
## 2. Flag Naming Convention
15+
16+
All rule-related flags will follow a consistent pattern: `--rule-<type>-<property>`. This convention is clear, avoids conflicts, and aligns with the `AGENTS.md` guidelines for flattening nested API objects.
17+
18+
- `<type>`: The type of the rule (e.g., `filter`, `retry`, `transform`).
19+
- `<property>`: The specific attribute of the rule to be configured (e.g., `body`, `strategy`, `name`).
20+
21+
**Example:**
22+
- `--rule-retry-strategy=linear`
23+
- `--rule-filter-body='{"$.type":"payment"}'`
24+
- `--rule-transform-name="my-transform"`
25+
26+
## 3. Rule Ordering Mechanism
27+
28+
The execution order of the rules is determined by the order in which the rule-related flags appear in the command.
29+
30+
- The **first occurrence of a flag for a specific rule type** (e.g., the first `--rule-filter-*` flag) establishes that rule's position in the execution order.
31+
- Subsequent flags for the **same rule type** will modify the existing rule object without changing its position.
32+
- Only **one rule of each type** is supported, as per the API constraints. If multiple flags for the same property are provided (e.g., two `--rule-retry-count` flags), the command will fail validation.
33+
34+
**Example:**
35+
```bash
36+
hookdeck connection create \
37+
--rule-filter-body '{"$.type":"payment"}' \ # 1. Filter rule is first
38+
--rule-transform-name "my-transform" \ # 2. Transform rule is second
39+
--rule-filter-headers '{"$.auth":"present"}' # Modifies the first rule
40+
```
41+
This command will generate a `rules` array where the `filter` rule is at index 0 and the `transform` rule is at index 1.
42+
43+
## 4. Complete Flag List by Rule Type
44+
45+
### 4.1 Retry Rule
46+
47+
| Flag | Type | Description |
48+
|---|---|---|
49+
| `--rule-retry-strategy` | `string` | The retry strategy. Accepted values: `linear`, `exponential`. |
50+
| `--rule-retry-count` | `integer` | The number of retry attempts. |
51+
| `--rule-retry-interval` | `integer` | The interval between retries in milliseconds. |
52+
| `--rule-retry-response-status-codes` | `string` | Comma-separated list of HTTP status codes to retry on (e.g., "500-599,!401,404"). |
53+
54+
### 4.2 Filter Rule
55+
56+
| Flag | Type | Description |
57+
|---|---|---|
58+
| `--rule-filter-body` | `string` | A JQ expression to filter on the request body. |
59+
| `--rule-filter-headers` | `string` | A JQ expression to filter on the request headers. |
60+
| `--rule-filter-query` | `string` | A JQ expression to filter on the request query parameters. |
61+
| `--rule-filter-path` | `string` | A JQ expression to filter on the request path. |
62+
63+
### 4.3 Transform Rule
64+
65+
| Flag | Type | Description |
66+
|---|---|---|
67+
| `--rule-transform-name` | `string` | The name or ID of the transformation to apply. |
68+
| `--rule-transform-code` | `string` | The transformation code (if creating inline). |
69+
| `--rule-transform-env` | `string` | A JSON string representing environment variables for the transformation. |
70+
71+
### 4.4 Delay Rule
72+
73+
| Flag | Type | Description |
74+
|---|---|---|
75+
| `--rule-delay-delay` | `integer` | The delay in milliseconds. |
76+
77+
### 4.5 Deduplicate Rule
78+
79+
| Flag | Type | Description |
80+
|---|---|---|
81+
| `--rule-deduplicate-window` | `integer` | Time window in milliseconds for deduplication. |
82+
| `--rule-deduplicate-include-fields` | `string` | Comma-separated list of fields to include for deduplication. |
83+
| `--rule-deduplicate-exclude-fields` | `string` | Comma-separated list of fields to exclude for deduplication. |
84+
85+
86+
## 5. Validation Rules
87+
88+
Validation will be performed in the `PreRunE` phase of the Cobra command to ensure that the combination of flags is valid before making an API call.
89+
90+
### 5.1 General Rules
91+
- If any `--rule-*` flag is used, the corresponding rule object will be constructed.
92+
- A maximum of one rule per type is allowed. The CLI will fail if, for example, a retry rule is defined using both flags and the JSON fallback.
93+
94+
### 5.2 Per-Type Validation
95+
96+
- **Retry Rule**:
97+
- If any `--rule-retry-*` flag is provided, `--rule-retry-strategy` is required.
98+
- `--rule-retry-strategy` must be one of `linear` or `exponential`.
99+
- `--rule-retry-count` and `--rule-retry-interval` must be positive integers.
100+
101+
- **Filter Rule**:
102+
- If any `--rule-filter-*` flag is provided, at least one of `--rule-filter-body`, `--rule-filter-headers`, `--rule-filter-query`, or `--rule-filter-path` must not be empty.
103+
104+
- **Transform Rule**:
105+
- If any `--rule-transform-*` flag is provided, `--rule-transform-name` is required.
106+
- `--rule-transform-env` must be a valid JSON string if provided.
107+
108+
- **Delay Rule**:
109+
- `--rule-delay-delay` must be a positive integer.
110+
111+
- **Deduplicate Rule**:
112+
- If any `--rule-deduplicate-*` flag is provided, `--rule-deduplicate-window` is required.
113+
- `--rule-deduplicate-window` must be a positive integer.
114+
115+
## 6. JSON Fallback Strategy
116+
117+
For complex configurations or for users who prefer to manage rules as code, a JSON fallback mechanism will be provided. This aligns with the existing strategy for source and destination configurations.
118+
119+
| Flag | Type | Description |
120+
|---|---|---|
121+
| `--rules` | `string` | A JSON string representing the entire `rules` array. |
122+
| `--rules-file` | `string` | Path to a JSON file containing the `rules` array. |
123+
124+
**Behavior:**
125+
- If `--rules` or `--rules-file` is used, all individual `--rule-*` flags will be ignored.
126+
- The JSON provided must be a valid array of rule objects, conforming to the Hookdeck API schema.
127+
- The order of the rules in the JSON array will be preserved.
128+
129+
130+
## 7. Usage Examples
131+
132+
### Example 1: Simple Retry and Filter
133+
This command creates a connection that first retries failed webhooks, then filters them.
134+
```bash
135+
hookdeck connection create \
136+
--source-name "my-source" \
137+
--destination-name "my-destination" \
138+
--rule-retry-strategy="exponential" \
139+
--rule-retry-count=3 \
140+
--rule-filter-body='{"$.event_type":"user.created"}'
141+
```
142+
**Resulting `rules` array:**
143+
```json
144+
[
145+
{"type": "retry", "strategy": "exponential", "count": 3},
146+
{"type": "filter", "body": "{\"$.event_type\":\"user.created\"}"}
147+
]
148+
```
149+
150+
### Example 2: Filter with Multiple Properties
151+
The filter rule is defined by the first `--rule-filter-*` flag. Subsequent flags for the same rule amend the original rule object.
152+
```bash
153+
hookdeck connection create \
154+
--source-name "my-source" \
155+
--destination-name "my-destination" \
156+
--rule-filter-headers='{"$.content-type":"application/json"}' \
157+
--rule-transform-name="my-transform" \
158+
--rule-filter-body='{"$.status":"completed"}'
159+
```
160+
**Resulting `rules` array:**
161+
```json
162+
[
163+
{
164+
"type": "filter",
165+
"headers": "{\"$.content-type\":\"application/json\"}",
166+
"body": "{\"$.status\":\"completed\"}"
167+
},
168+
{"type": "transform", "transformation_id": "my-transform"}
169+
]
170+
```
171+
172+
### Example 3: Using the JSON Fallback
173+
This example uses a JSON file to define the rules, which is useful for complex configurations.
174+
```bash
175+
# rules.json
176+
# [
177+
# {"type": "delay", "delay": 5000},
178+
# {"type": "retry", "strategy": "linear", "count": 5, "interval": 10000}
179+
# ]
180+
181+
hookdeck connection create \
182+
--source-name "my-source" \
183+
--destination-name "my-destination" \
184+
--rules-file="rules.json"
185+
```
186+
**Resulting `rules` array:**
187+
```json
188+
[
189+
{"type": "delay", "delay": 5000},
190+
{"type": "retry", "strategy": "linear", "count": 5, "interval": 10000}
191+
]
192+
```
193+
194+
## 8. Edge Cases
195+
196+
- **Duplicate Rule Types**: If a user specifies a rule type with both flags and the JSON fallback (e.g., `--rule-retry-strategy` and `--rules` containing a retry rule), the CLI will exit with an error, as the JSON fallback takes precedence and all individual flags are ignored.
197+
- **Conflicting Properties**: If a user provides the same flag twice (e.g., `--rule-retry-count=3 --rule-retry-count=5`), Cobra will automatically use the last value provided. Our validation will not need to handle this explicitly.
198+
- **Empty Rule Properties**: If a flag is provided with an empty value (e.g., `--rule-filter-body=""`), it will be treated as an empty string and passed to the API. The API will then perform its own validation.
199+
200+
## 9. Implementation Guidance
201+
202+
This section provides guidance for the `code` mode to implement this design.
203+
204+
### 9.1 Flag Parsing and Rule Construction
205+
- Use a custom `RuleSet` struct to manage the rules as they are parsed. This struct should maintain the order of the rules and the properties of each rule.
206+
- Iterate through the command's flags in the `PreRunE` function. When a `--rule-*` flag is encountered, add the rule to the `RuleSet`.
207+
- The `RuleSet` should have a method to convert the rules into the JSON format expected by the API.
208+
209+
### 9.2 Cobra Flag Definition
210+
- Define all `--rule-*` flags in the `connection_create.go` and `connection_update.go` files.
211+
- Use `pflag` to access the flags in the order they were set on the command line. This is crucial for preserving the rule order.
212+
213+
### 9.3 Validation Logic
214+
- Implement the validation rules defined in Section 5 within the `PreRunE` function.
215+
- Provide clear and actionable error messages to the user if validation fails, as per `AGENTS.md`.
216+
217+
### 9.4 API Client
218+
- Update the `ConnectionCreate` and `ConnectionUpdate` functions in the API client to accept the `rules` array.
219+
- Ensure that the API client correctly serializes the `rules` array into the request body.

0 commit comments

Comments
 (0)