Scanning and detecting objects
1. Setting up a scanning session
1.1 Open BP_ARPawn and set the ARsession variable to ObjectScanning
1.2 Go to project settings\platforms\IOS and make sure this line is in the additional PList data box:
<key>UIFileSharingEnabled</key><string>true</string>
This will allow iTunes to access the scanned object files that you will save on your device.
DO NOT check Supports iTunes File Sharing, leave it UNCHECKED
1.3 Package the project for development and load the .ipa file onto your device through iTunes
or Launch to your device
2. Scanning your objects
2.1 Prepare your objects to be scanned using these guidelines:
• Ensure that the object is well-lit from all sides.
• Provide a light temperature of around ~6500 Kelvin (D65) – similar with daylight. Avoid warm or any other
coloured light sources.
• Set the object in front of a matte, middle-grey background.
• Position the object you want to scan on a surface free of other objects (like an empty tabletop.)
• Choose Objects with non-repetitive textures
• Avoid transparent objects like a glass statuette or jar of water.
• Avoid highly reflective objects like mirror or chrome sphere.
• Avoid objects with a chromatic dispersion effect like surface of DVD or precious stones in jewellery.
2.2 Open the app on your device and start scanning an object from different angles, you will see this
dialog box appear
Touch the axis arrows to reposition the scanning box
2.3 Touch the Scale button to resize the scanning box
2.4 Touch the Scan button when you’ve captured plenty of feature points and then adjust where you want
the origin point to be located.
2.5 Touch the Test button to check your object can be detected and the origin point is in the correct
position
2.6 If you are happy with the results, enter the name of the object and Touch the Save button
2.7 You should then see a message confirming the object has been saved
2.8 Repeat this step for each object you want to scan
3. Accessing the save game file
3.1 Plug your device into your computer and open iTunes, then click the phone symbol
3.2 Click file sharing, then select the UE4 app and the project folder, then click save
3.3 Once the folder has been saved from iTunes open it and locate the .sav files.
(This contains all the object information saved in the scanning session)
3.4 Copy the files and place them in the SaveGames folder in your UE4 project directory.
If this folder does not already exist create a new one in the Saved folder.
4. Extracting the scanned object data
4.1 Open the ReadObjectDataMap
4.2 Click on BP_ARPawn in the level and create a new array item for each scanned object and enter the
name of your .sav files
4.3 Click Read Save Game to load the scanned object data
4.4 expand the Scanned Object List to check it has been populated
4.5 Collapse the Objects List then right click and copy it
4.6 In BP_ARPawn click on the Scanned Object List variable then in the details panel (right hand side)
right click on Objects and paste
4.7 Expand the Objects list and check the scanned object variables were copied successfully
4.8 Repeat the last 4 steps for each scanned object
4.9 Go to the Candidate Objects folder
4.10 Add a new Data Asset for each scanned object
4.11 Select ARCandidateObject and give it the same name as your scanned object
4.12 Go back to BP_ARPawn and add the newly created candidate objects to CandidateObjects array
variable
(These Candidate Objects will be populated with the scanned object data at runtime and added to the
session config ready for detection)
5. Detecting Objects
5.1 Go to BP_ARPawn and change the ARsession variable to ObjectDetectionSession
5.2 In BP_ARPawn make sure Config variable is set to ObjectDetectionSessionConfig
5.3 When the object is detected it will spawn a label blueprint that displays the name of the object as 3D
text above it.
You can change what happens when the object is detected by clicking on the node graphs below to
get to the code. You can change the blueprint that’s spawned or add a custom event, whatever
functionality you want to add this is where it can be added
5.4 Save all changes and package the game to test on your device
You should see a debug message with the name of your object, the bounding box size and a
confirmation message that the session has started
Move your device around the object to test it, after a few seconds the label should appear.
