Skip to content

feat: add JSONPathExtractor tool #5052

Merged
HenryHengZJ merged 5 commits intoFlowiseAI:mainfrom
anatolyburtsev:feature/json-path-extractor
Aug 18, 2025
Merged

feat: add JSONPathExtractor tool #5052
HenryHengZJ merged 5 commits intoFlowiseAI:mainfrom
anatolyburtsev:feature/json-path-extractor

Conversation

@anatolyburtsev
Copy link
Copy Markdown
Contributor

Summary

  • Add JSONPathExtractor tool for extracting specific values from JSON data
  • It uses lodash.get under the hood
  • added unit tests and configured jest for packages/components

Motivation

When building AI workflows in Flowise, users often need to extract specific values from API responses or structured data. Currently, there's no built-in tool to easily parse JSON and extract nested values without writing custom code.

Use Cases

  1. API Response Parsing: Extract specific fields from complex API responses
  2. Data Transformation: Pull out nested values from webhook payloads
  3. LLM Output Processing: Extract structured data from LLM JSON responses

Example Usage

Consider an e-commerce workflow that needs to process order data from an API:

  {
    "order": {
      "id": "ORD-12345",
      "customer": {
        "name": "John Doe",
        "email": "[email protected]"
      },
      "items": [
        {
          "product": "Laptop",
          "price": 999.99,
          "quantity": 1
        }
      ],
      "shipping": {
        "address": {
          "city": "San Francisco",
          "country": "USA"
        }
      }
    }
  }

With JSONPathExtractor, you can easily extract:

  • Customer email: order.customer.email → "[email protected]"
  • First item price: order.items[0].price → 999.99
  • Shipping city: order.shipping.address.city → "San Francisco"

Implementation Details

  • Lodash-based extraction: Uses lodash.get for reliable path resolution
  • Flexible input: Accepts JSON strings, objects, or arrays
  • Error handling: Configurable returnNullOnError parameter to either throw errors or return null
  • Edge case support: Handles numeric string keys, array indexing, special characters in keys

Test Coverage

Added unit tests to cover:

  • Basic and nested path extraction
  • Array indexing and multi-dimensional arrays
  • Primitive value handling (strings, numbers, booleans, null)
  • Special characters in keys (spaces, dashes, unicode, numeric strings)
  • Error handling modes (throw vs return null)
  • Complex nested structures

Result of the run:

flowise-components:test: PASS nodes/tools/JSONPathExtractor/JSONPathExtractor.test.ts (5.045 s)
flowise-components:test:   JSONPathExtractor
flowise-components:test:     Tool Initialization
flowise-components:test:       ✓ should throw error when path is not provided (7 ms)
flowise-components:test:       ✓ should initialize tool with path and default returnNullOnError (1 ms)
flowise-components:test:       ✓ should initialize tool with custom returnNullOnError
flowise-components:test:     JSONPathExtractorTool Functionality
flowise-components:test:       Positive test cases - Path extraction
flowise-components:test:         ✓ should extract simple path from object
flowise-components:test:         ✓ should extract nested path from object (1 ms)
flowise-components:test:         ✓ should extract array index access
flowise-components:test:         ✓ should extract multi-dimensional array
flowise-components:test:         ✓ should extract object return (stringified)
flowise-components:test:         ✓ should extract array return (stringified) (1 ms)
flowise-components:test:         ✓ should extract deep nesting
flowise-components:test:         ✓ should extract array at root with index
flowise-components:test:       Primitive value handling
flowise-components:test:         ✓ should handle string value
flowise-components:test:         ✓ should handle number value
flowise-components:test:         ✓ should handle zero value
flowise-components:test:         ✓ should handle boolean true value (1 ms)
flowise-components:test:         ✓ should handle boolean false value
flowise-components:test:         ✓ should handle null value
flowise-components:test:         ✓ should handle empty string value
flowise-components:test:       Special characters in keys
flowise-components:test:         ✓ should handle dashes in keys
flowise-components:test:         ✓ should handle spaces in keys
flowise-components:test:         ✓ should handle unicode in keys
flowise-components:test:         ✓ should handle numeric strings in keys
flowise-components:test:       Error handling - throw mode
flowise-components:test:         ✓ should throw error for path not found (4 ms)
flowise-components:test:         ✓ should throw error for invalid JSON string
flowise-components:test:         ✓ should throw error for array index on object
flowise-components:test:         ✓ should throw error for out of bounds array
flowise-components:test:       Error handling - null mode
flowise-components:test:         ✓ should return null for path not found
flowise-components:test:         ✓ should return null for invalid JSON string (1 ms)
flowise-components:test:         ✓ should return null for null in path
flowise-components:test:         ✓ should return null for empty array access
flowise-components:test:         ✓ should return null for property on primitive
flowise-components:test:         ✓ should still extract valid paths when returnNullOnError is true
flowise-components:test:       Complex structures
flowise-components:test:         ✓ should handle deeply nested arrays and objects
flowise-components:test:
flowise-components:test: Test Suites: 1 passed, 1 total
flowise-components:test: Tests:       33 passed, 33 total
flowise-components:test: Snapshots:   0 total
flowise-components:test: Time:        5.151 s
flowise-components:test: Ran all test suites.

Screenshot of tool configuration page

Screenshot 2025-08-10 at 11 16 24 PM

  - Implement JSONPathExtractor tool for extracting values from JSON using path notation
  - Use lodash.get for robust path extraction supporting edge cases (numeric string keys, array indexing)
  - Add configurable error handling with returnNullOnError parameter
  - Include comprehensive test suite with 34 tests covering all scenarios
  - Support JSON strings, objects, and arrays as input
@anatolyburtsev anatolyburtsev force-pushed the feature/json-path-extractor branch from 8dd3929 to 418299b Compare August 11, 2025 06:26
@HenryHengZJ
Copy link
Copy Markdown
Contributor

love these super useful tools you have added! and the additional of the unit test

let me know if you need help on resolving the pnpm lock conflict

@anatolyburtsev
Copy link
Copy Markdown
Contributor Author

Sure! Thanks for switch reply!

@anatolyburtsev
Copy link
Copy Markdown
Contributor Author

@HenryHengZJ Mind to check it and wrap it up? Thanks

Prevents test files from being included in the dist folder which was causing
"jest is not defined" errors during server startup.
@anatolyburtsev
Copy link
Copy Markdown
Contributor Author

Fixed warning about missing jest when server start.

@HenryHengZJ
Copy link
Copy Markdown
Contributor

thanks!

@HenryHengZJ HenryHengZJ merged commit b126472 into FlowiseAI:main Aug 18, 2025
2 checks passed
erhhung pushed a commit to erhhung/flowise that referenced this pull request Oct 5, 2025
* feat: add JSONPathExtractor tool with lodash-based path extraction

  - Implement JSONPathExtractor tool for extracting values from JSON using path notation
  - Use lodash.get for robust path extraction supporting edge cases (numeric string keys, array indexing)
  - Add configurable error handling with returnNullOnError parameter
  - Include comprehensive test suite with 34 tests covering all scenarios
  - Support JSON strings, objects, and arrays as input

* fix lint

* Update pnpm-lock.yaml

* fix: exclude test files from TypeScript compilation

Prevents test files from being included in the dist folder which was causing
"jest is not defined" errors during server startup.

---------

Co-authored-by: Henry Heng <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants