Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Efficient Detection of Interactive Elements using computed cursor style #1031

Merged
merged 1 commit into from
Mar 25, 2025
Merged

Efficient Detection of Interactive Elements using computed cursor style #1031

merged 1 commit into from
Mar 25, 2025
+17 −3

Conversation

Rahul-Sharma-1729
Copy link

Background

In most web applications, developers or frameworks provide visual cues to indicate interactive elements. One common approach is changing the cursor style when hovering over such elements. This helps users quickly identify elements they can interact with.

Upon analyzing widely used websites such as Amazon, Google, eBay, Salesforce, Reddit, Google Docs etc., I observed a consistent pattern: interactive elements typically have cursor styles that differ from the default pointer, signaling interactivity.

Implementation

To enhance this behavior, I have implemented the doesElementHaveInteractivePointer function. This function determines whether an element has an interactive cursor by inspecting its computed cursor style via window.getComputedStyle().

Key advantages of this approach:

  • Performance Optimization: It eliminates the need for more expensive DOM traversal operations (e.g., closest()) and avoids calls to window.getEventListeners().
  • Efficiency: By directly checking the computed styles, it removes the need to create additional data structures such as sets, improving execution speed.
  • Additionally removed the unused call to window.getComputedStyle() in the same function.

This implementation ensures a lightweight and efficient way to detect interactive elements, aligning with best practices in web development.

@RsRahulPro
Update isInteractiveElement handling:
cdae535 
- Remove unused window.getComputedStyle call.
- Add a much faster check for interactive element using cursor pointer which cut short the function for 90% of the cases.
@CLAassistant
Copy link

CLAassistant commented Mar 15, 2025 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@Rahul-Sharma-1729
Copy link
Author

Upon Testing the new changes on some commonly used websites internationally, these are the results I found:

  1. The original function misses some interactive elements on the page.
  2. The newer function captures more elements on a granular level, with better average time per element and a better "slowest time"
  3. The test was done by getting all the elements on the page using document.querySelectorAll("*") and traversing through the array of elements to run the function and find the metrics attached.
  4. Manual verification was also done for each interactive element captured by logging it and mapping with the actual website.

@MagMueller @gregpr07
image

@teddybear082
Copy link

I adapted this code change to a similar project and seems to work great. Good job OP.

@Rahul-Sharma-1729
Copy link
Author

@teddybear082 Hey thanks teddy, This code is tried and tested on production environments as well, hopefully it can be merged to browser-use

@gregpr07
Copy link
Collaborator

Oh wow, that's really smart!! Thanks man

@pirate pirate merged commit d51f43f into browser-use:main Mar 25, 2025
1 check passed
@pirate
Copy link
Member

pirate commented Mar 25, 2025

@Rahul-Sharma-1729 thanks again for this. Out of curiosity what is your day job? Do you work on browser automation?

@Rahul-Sharma-1729
Copy link
Author

Rahul-Sharma-1729 commented Mar 26, 2025 •
edited
Loading

@pirate Hey, I'm a software engineer at an AI startup.
Yeah, I do work on browser automation—most of my day-to-day work involves browsers, web extensions, data scraping, and similar things. I spend a lot of time dealing with XPaths, shadow DOMs, Iframes, and all that fun stuff on daily basis.

@pirate
Copy link
Member

pirate commented Mar 26, 2025 •
edited
Loading

nice nice, well if you're ever looking for work or interested in switching you should DM me: x.com/theSquashSH 😉

@pirate pirate changed the title Performance Improvement: Efficient Detection of Interactive Elements Efficient Detection of Interactive Elements using computed cursor style Apr 1, 2025
This was referenced Apr 1, 2025
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pirate pirate pirate left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@Rahul-Sharma-1729 @CLAassistant @teddybear082 @gregpr07 @pirate @RsRahulPro