Image & Text Automation
Image & Text Automation - Agenda
How to automate virtualized
applications
Automation Techniques Best Practices
Image Q&A
Text Test
Keyboard automation
Online Resources
Insert data
Retrieve data
How to Automate Virtualized Apps
Things you should know
UI Elements are not available in Citrix.
Everything appears like an image.
App must always be visible on screen.
Only working with the main window/element.
Image Automation
Input Output
Activities Activities
Click Image used to identify Not Applicable
elements based on its image
Find Image used to verify the
active window
Insert data in Citrix
Click relative to image or text For instance in this application we indicate with
yellow the text and with red the area which we
want to be clicked.
Keyboard hotkeys, tab, enter
Paste text via shared clipboard
Retrieve data
Copy selected text from text areas, tables and so on.
Click relative to a static text and then get the
data in clipboard
Retrieve data
Scrape - relative to image or text
Example: Scrape relative to “Cash In”.
Text Automation
Input Output
Activities Activities
● Click Text used to identify ● Get Text used to identify
elements based on its image elements based on its image
● Find Image used to verify the
active window
OCR Engines
Filter Better on
More Preffered Can invert Set expected
allowed Microsoft
languages area size colors text format
characters Fonts
Google
Tesseract
can add small ✓ ✓ ✓
Microsoft
Modi
default large ✓
Keyboard Automation
Input Output
Activities Activities
Send Hotkey Select Text
Type - send keyboard commands Type: "[k(end)d(shift)d(home)u(shift)u(home)]"
Copy Selected Text
Review
Input Output
Image Yes No
Text Yes Yes
Keyboard Yes Yes
Identify elements
Keyboard automation
Sometimes could be very difficult to do a good text/image automation so you
can use the keyboard.
Attach to the window.
Use “Send Hotkeys” activity.
Identify Elements
Text - identified by specific text/word regions
1) OCR - 98% accurate
▪ Google OCR Engine
▪ Microsoft OCR Engine
2) Native - 100% accurate
Best practices
Be careful when you try to click on a text to be unique. For instance if you
want to click on a “Cancel” button and the text “cancel” is somewhere else
then you better use click image and indicate that button.
The text/image always should be visible. In Citrix automation if the user is
not able to see an image or a text neither the robot.
Feel free to use .NET methods like “Split()” , “Substring()”, “ StartsWith() ”
and so on when you scrape a text.
Exercise 1: add new expense
Download Expenseit application and add a new expense in the application
using only image/text and hotkeys.
1. Download the application from here:
https://drive.google.com/file/d/0BxVAKI3l9Pc5d0JqY2E2RGYtZDA/vie
w?usp=sharing
2. Add expenses rows just by using text or image techniques
Online Resources
Data entry automation in Citrix
http://www.uipath.com/tutorials/citrix-automation-data-entry
Retrieve data using clipboard
http://www.uipath.com/tutorials/citrix-automation-keyboard-hotkeys
Retrieve data using OCR
http://www.uipath.com/tutorials/citrix-automation-ocr-data-extraction
Automation using text scraping
http://www.uipath.com/tutorials/sap-gui-automation-data-entry-scraping
