MineSight® Grail—Data Access and Embedded Scripts
21st
Annual MineSight® Grail—Data Access and Embedded Script
Seminar
Introduction .................................................................................................................................. 1
How Python Integrates with MineSight® 3-D........................................................................ 2
Data Access Examples ................................................................................................................. 2
Models ....................................................................................................................................... 2
Geometry ................................................................................................................................... 4
Drillholes .................................................................................................................................. 5
MineSight® Grail Inside MineSight® 3-D ............................................................................... 6
Common Terms........................................................................................................................ 6
Working with the Selection Buffer ...................................................................................... 7
Manipulating the Datamanager............................................................................................ 8
Automating Triangulations ................................................................................................... 9
Automating Contouring ....................................................................................................... 10
Future Plans................................................................................................................................. 11
Summary...................................................................................................................................... 11
Introduction
MineSight® Grail is a flexible and powerful scripting library for customizing MineSight®
for site implementation engineers and end users. In addition, MineSight® Grail is a
replacement for MINTEC, inc.’s traditional MineSight® Compass™ procedure language.
It exploits the richness of Python as a scripting interface and provides a flexible GUI
layout tool. MineSight® Grail procedures can be run inside of MineSight® Compass™,
use MineSight® Compass™ Multi-Run, run inside MineSight® 3-D, as well as executed in
standalone mode from a desktop shortcut or from a command window.
There are many instances when working with a MineSight® project that a user is
required to customize and automate a number of processes. The current customization
method is to use MineSight® Compass™ Procedures, which are based on a proprietary
language developed by MINTEC over a number of years. While this has served our
clients well, extending the traditional MineSight® Compass™ Procedure language is non-
trivial. Since we already have developed Python scripting experience in providing user
customizable reserve calculations in the MineSight® Interactive Planner, it was decided to
extend the use of Python into a more general scripting interface into the system.
To accommodate this goal, we required the new scripting system to:
• provide extensible, powerful scripts that can interact with outside processes
• provide increased GUI composition capabilities
• have improved ease of use and development
• provide a new framework for scripting in MineSight® Compass™ and
MineSight® 3-D
• have backward compatibility with existing MineSight® Compass™ procedures,
including multi-run capability
• have improved GUI look and feel, as compared to traditional
MineSight® Compass™ procedures
• allow access to all important MineSight® data
Starting with MineSight® Grail v.1.10 the MineSight® Interactive Planner tool will be
integrated with the MineSight® Grail scripting library to provide a single location for all
your Python and MineSight® scripting needs.
How to Become a Better MineSight® User
Page 1
MineSight® Grail—Data Access and Embedded Scripts
21st
It is important to note that MineSight Grail is not just a replacement for MineSight
® ®
Annual
Compass™ procedures, but is a ‘glue’ interface that lets us bind processes together in an Seminar
easy way, and provide a rich interface toolkit that can be embedded in or run independent
of MineSight® 3-D or MineSight® Compass™.
This paper will explore two major concepts that MineSight® Grail brings to MineSight®.
They are:
1. Open data access
The data that MineSight® programs and procedures store are now accessible via
MineSight® Grail. This paper will concentrate on working with Model, Geometry
MSR, and Drillhole data files.
2. MineSight® Grail Embedded
This is the phrase we use to refer to running Python scripts and using the
MineSight® Grail library to help automate tasks within MineSight® 3-D. In addition,
this will also allow on-site engineers to customize functions MineSight® 3-D
functionality to help meet their unique circumstances. This will be available in
version 3.4 of MineSight® 3-D.
How Python Integrates with MineSight® 3-D
The popular scripting language, Python, integrates with MineSight® 3-D via the
MineSight® Grail library.
The MineSight® Grail library provides python modules that allow,
• Direct access to MineSight® data files.
• Python script execution within a Multi-run context.
• Work with MineSight® Interactive Planner.
• Access to the internal operations of MineSight® 3-D.
• The ability to build graphical user interfaces using Python.
Combining the power of the Python scripting language with MineSight® Grail library
will allow one to customize MineSight® products for site-specific implementations.
Data Access Examples
Since the release of MineSight® Grail v1.0 the most popular feature has been the access to
the MineSight® data files (MSR). The flexibility of the Python scripting language combined
with the MineSight® Grail library allows you perform custom on-site calculations using the
MineSight® data files to suit a particular business need.
We will focus on three major types of MineSight® data files,
1. Models
Provides read and write access to the 3-D model files.
2. Geometry
Provides read and write access to the MineSight® 3-D geometry files.
3. Drillhole
Provides read and write access to the drill hole files.
With these data files you can make your calculations and use MineSight® 3-D to
view the results.
The follow sections will review some of the common operations that can be performed
using MineSight® Grail and Python.
Models
Prior to MineSight® Grail v1.0 access to 3-D model data could only be done via
FORTRAN subroutines or through RP rountines. Now, with MineSight® Grail’s
grail.data.model module you can access any 3-D block model file via a Python script.
How to Become a Better MineSight® User
Page 2
MineSight® Grail—Data Access and Embedded Scripts
21st
Annual One of the major benefits with the MineSight® Grail approach to accessing data is that
you can retrieve what are known as slabs from the 3-D model. These slabs are chunks of
Seminar cells that are defined by you when you extract the data. You are no longer limited to small
sets of model cells when working with 3-D models files, you can specify how you want to
traverse the model, and find the representation that best suites your needs.
Lets work through an example of model access to illustrate how to make a basic
computation for each cell in the model. To start with, we will import the model module
and define the dimensions and items that we wish to work with,
from grail.data import model
lvlmin, lvlmax = 1, 10
rowmin, rowmax = 1, 500
colmin, colmax = 1, 500
items = [“CU”, “MOLY”, “EQ”]
We are going to traverse the model and use the model items CU, MOLY and EQ. We
will assume that the “samp10.dat” and “samp15.dat” files have the above dimensions and
items. You can always use the grail.data.pcf module to query the PCF file and determine
how the model file is formed. Now, lets set up our looping calculation.
# model access “grain” is row by row
for row in xrange(rowmin, rowmax+1):
# define and retrieve a “slab”
m = model.Model(“samp10.dat”, “samp15.dat”,
lvlmin, lvlmax,
row, row,
colmin, colmax,
items)
slab = m.slab()
for col in xrange(colmin, colmax+1):
for lvl in xrange(lvlmin, lvlmax+1):
# make our computation; if either MOLY or CU
# are missing, set EQ to missing.
moly = slab.modget(“MOLY”, lvl, row, col)
cu = slab.modget(“CU”, lvl, row, col)
if model.isdefined(moly) and model.isdefined(cu):
eq = moly*0.5 + cu*1.2
else:
eq = model.UNDEFINED
slab.modset(“EQ”, lvl, row, col, eq)
m.storeslab() # commit our changes back to the model.
m.free() # free up some underlying memory.
The two important features to note here are the “grain” of the model access, and the use
of modget() and modset() for the access to the Model.
When we mention the “grain” of a model, we are referring to the most efficient method
for model access. In this case the fastest way to access the model is to define your slab on
a row-by-row basis, and then walk through the cells inside that row. However, you are
in no way limited to this scheme of iteration. The way you decide to access the model is
completely up to you, and dependent on your particular problem. Note that since you can
now retrieve a “slab”, you should be aware of the memory consumption issue of reading
a slab that represents an entire model, and all the items. In many situations, that could be
a problem. It is better to structure your script so that a slab is pulled bench at a time, or
section at a time, or if you need bench to bench calculations, three benches at a time.
How to Become a Better MineSight® User
Page 3
MineSight® Grail—Data Access and Embedded Scripts
21st
The modset() and modget() methods on the slab object, are new in MineSight Grail
®
Annual
v1.1. Previously in v1.0 the indices for the slab object where always offset from 0 and went Seminar
to the maximum extent of the slab’s definition. For example, if you decided to extract a
slab with 5 by 5 by 5 dimensions from the middle of a Model file (say at 200, 200, 200),
then you would iterate across the slab from 0 to 4 in all dimensions. Where the slab at
0, 0, 0 would correspond to 200, 200, 200 in the model. This behaviour is still preserved
when you use the square brackets, [], or use the get() and set() methods on the slab object.
However, over the last year this concept, rightfully, proved confusing to our users. To
alleviate the confusion we introduced the modset() and modget() methods. These methods
allow you access the slab with the exact same dimensions that you used to specify the slab.
For example, if you extract a slab that is 5 by 5 by 5 from the 200, 200, 200 location in the
Model file, then you would use 200, 200, 200 to access the first cell in the Slab object using
modset() and modget().
Geometry
The geometry MSR files contain 3-D information for MineSight® 3-D. With MineSight®
Grail v1.0 the grail.data.geometry module provides access to the 3-D (and 2-D) data stored
within the MSR.
An example of accessing the MSR is the truck.py script (shown below). This script will
read XYZ co-ordinate information and update the position of a label in a geometry file.
Combining this with the timed refresh option will allow you to view live truck movements
in your 3-D geometry. This script could easily be extended to using Dispatch’s add-on
called Powerview and allow you to get the truck position directly from an SQL database.
The truck.py script is as follows:
import sys, string, fileinput, time
from grail.data import geometry
def isgeometryfile(msrpath):
“””attempt to open the file, if it works, then its an msr file.”””
try:
msr = geometry.Geometry(msrpath)
except GeometryError:
return 0 # not an msr
msr.free()
return 1 # it was a msr
def readfile(pointfile,coords):
for i in fileinput.input(pointfile):
line = string.split(i)
x1 = string.atof(line[0])
y1 = string.atof(line[1])
z1 = string.atof(line[2])
CentList = [(x1,y1,z1),(x1,y1-1,z1),(x1-1,y1,z1)]
coords.append(CentList)
return coords
def markerdisplay(msrpath,coords):
msr = geometry.Geometry(msrpath)
for i in range(len(coords)):
msr.addsemitlabel([“Truck 1”],coords[i])
print coords[i]
msr.storeresource()
How to Become a Better MineSight® User
Page 4
MineSight® Grail—Data Access and Embedded Scripts
21st
Annual time.sleep(5) # time it waits before reading next point
msr.deletesemitlabelat(0)
Seminar msr.free() # Clean up pythons internal structure
def main():
coords = []
try:
# attempt to read the paths from the command line.
msrpath = sys.argv[1]
pointfile = sys.argv[2]
except IndexError, e:
print “error: require msrpath and pointfile path.”
if isgeometryfile(msrpath):
readfile(pointfile,coords)
markerdisplay(msrpath,coords)
if __name__==”__main__”:
main()
In this script, the user will enter a msrpath and a pointfile path when you start the script.
The script should be executed on a timer (using a Microsoft® Window’s task manager). The
script will go through the following steps,
• Read the point file to determine the trucks current location.
• Open the geometry file.
• Remove the old truck location, if one exists.
• Write the new truck location based on the point file.
Now, when MineSight® 3-D refreshes the Datamanager the new label location for the
truck will have changed. Using this script to retrieve truck location data from a database
would in effect show your truck positions live in a MineSight® 3-D project.
Drillholes
Starting in MineSight® Grail v1.15 the library will provide the ability to extract,
manipulate, and modify data in your drillhole and blasthole files. For simplicity, we will
refer to drillholes sets, but the concepts also extend to blasthole sets.
A drillhole set is analogous to a drillhole view in MineSight® 3-D.
The drillhole set is composed of three components,
1. drillhole or blasthole files: a pcf, an assay or composite file, and possibly a survey
file.
2. a method of naming the drillholes or blastholes.
3. a method of defining the drillhole or blasthole geometry.
All three of the above components must be specified when defining a drillhole set.
Methods are provided for defining (opening) a drillhole set, querying each of its
attributes, and reading and writing data to the drillhole files.
The following script illustrates looping across all drillholes in a sample drillhole set. For
each assay interval it will add the values for the items TOTCU and MOLY and store the
result in the XTRA1 item.
How to Become a Better MineSight® User
Page 5
MineSight® Grail—Data Access and Embedded Scripts
21st
from grail.data import dh Annual
# define a dhset Seminar
dhset = dh.opentypicaldhset(“samp10.dat”, “samp11.dat”, “samp12.dat”)
# get a list of the loaded dhs
dhs = dhset.loadeddhlist()
# loop over all the dhs in the set
for dh in dhs:
# make sure this dh has well-defined interval geometry
if dhset.dhbadintervalcount(dh) < 1:
# loop over the intervals in this dh. interval go from
# 1 --> dhintervalcount
for intv in range(1, dhset.dhintervalcount(dh)+1):
# retrieve this interval’s TOTCU and MOLY values
totcu = dhset.dhintervaldataat(dh, intv, “totcu”)
moly = dhset.dhintervaldataat(dh, intv, “moly”)
# if neither value is MISSING, add them and put the
# result in XTRA1. else put dh.MISSING in XTRA1
if totcu== dh.MISSING or moly== dh.MISSING:
xtra1 = dh.MISSING
else:
xtra1 = totcu + moly
dhset.dhintervalsetfloatdataat(dh, intv, “xtra1”, xtra1)
#store all the dhs
dhset.storealldhs()
# check a dh interval and make sure xtra1 = totcu + moly
# i happen to know this dh and interval are well-defined.
dh = “sm-021”
totcu = dhset.dhintervaldataat(dh, 6, “totcu”)
moly = dhset.dhintervaldataat(dh, 6, “moly”)
xtra1 = dhset.dhintervaldataat(dh, 6, “xtra1”)
print “dh = “, dh, totcu, “ + “, moly, “ = “, xtra1
MineSight® Grail Inside MineSight® 3-D
Currently MINTEC is working towards using MineSight® Grail to control the functions
of MineSight® 3-D. We expect this new feature to be available in MineSight® 3-D v3.4. The
process of embedding Python access to MineSight® 3-D via MineSight® Grail is known as
MineSight® Grail Embedded. With this new feature you will be able to gain access to such
components as the datamanager, the user selection set, and CAD tools.
The following sections will discuss some common examples of using MineSight® Grail
Embedded to execute tasks within MineSight® 3-D.
Common Terms
When talking about working with MineSight® Grail inside of MineSight® 3-D we have
some common terms. Some of these terms and definitions are:
• Elements
This is the generic term for all types of 3-D data that is used in MineSight® 3-D.
• Selection Buffer
This is a collection of elements that the user has highlighted as a selection.
How to Become a Better MineSight® User
Page 6
MineSight® Grail—Data Access and Embedded Scripts
21st
Annual • MineSight® Objects
These are the “files” inside the datamanager. A MineSight® object can be any of the
Seminar typical items that you work with when working with the datamanager.
Working with the Selection Buffer
The idea here is to allow you gain access to everything that a user has selected and
stored within the selection buffer. The user will select their elements, and then execute a
script that will work with the selected elements.
As an example, lets consider running a script that will look at each element within the
selection and report on its area. If the element does not have an area, for example a marker,
it will simple report “n/a” for not applicable. The script is as follows:
import sys
from grail import gsys
from grail import messages
from grail.ms3d import elementop
from grail.ms3d import selectionbuffer
from grail.ms3d import mssys
from grail.ms3d import project
# re-wire the “print” statement to the message window.
sys.stdout = mssys.stdout
def gmain(msg, data):
“””Main entry functin.”””
if msg is messages.gRUN:
areareport()
else:
return gsys.grailmain(msg,data)
def areareport():
elements = selectionbuffer.getelements()
if elements == []:
mssys.stdout.write(“There where no elements selected.”)
else:
reportareas(elements)
def reportareas(elements):
print “Element Name, Area”
cnt = 0
for element in elements:
try:
areastr = ‘%f’ % (elementop.calcarea(element))
cnt += 1
except TypeError, e: # element has no area (i.e. markers)
areastr = “n/a”
print “%s, %s %s^2” % (element.getname(),
areastr,
getprojectdim())
print “Computed %d areas.” % (cnt)
def getprojectdim():
if project.getunittype()==project.METRIC:
return “m”
else:
return “ft”
Early on in the script, we inform python that it will use the “mssys.stdout” for all its
print statements. Using mssys.stdout will ensure that all print statements are displayed in
How to Become a Better MineSight® User
Page 7
MineSight® Grail—Data Access and Embedded Scripts
21st
the MineSight 3-D Message window.
®
Annual
The elements are retrieved in the selectionbuffer.getelements() statement, and a loop Seminar
across all the elements is performed with the area calculated for all elements that can have
an area.
The area is then reported to the message window via the print statement.
Manipulating the Datamanager
With MineSight® Grail Embedded you will also be able to manipulate objects within the
datamanager. These objects are commonly referred to as MineSight® objects (MOBs).
Currently you can create a new MOB of type geometry, and populate it with geometry
objects. These technique will be used for the automated triangulation and contouring
examples also described in this paper.
However, as a simple example of creating a Datamanager object and inserting a
polylines consider the following script:
import sys
from grail import gsys
from grail import messages
from grail.ms3d import element
from grail.ms3d import mssys
from grail.ms3d import datamanager
# re-direct print statements to the message window.
sys.stdout = mssys.stdout
def gmain(msg, data):
if msg==messages.gRUN:
drawpolylines()
def drawpolylines():
# create our polygon elements to store in the
# geometry object.
numpolylines = 10
polylines = [None]*numpolylines
print “creating polylines...”,
for idx in range(numpolylines):
points = [[-10., -10., float(idx*10)],
[0., 0., float(idx*10)],
[10., 10., float(idx*10)]]
polylines[idx] = element.createpolyline(points)
print “OK.”
if datamanager.isobject(“\\newobject”):
print “removing \\newobject...”,
datamanager.remove(“\\newobject”)
print “OK.”
print “creating geometry object...”,
g = datamanager.create(“\\newobject”,
datamanager.GEOMETRY_OBJECT,
“\\materials\\Geometry”)
print “OK.”
print “adding polylines...”,
g.addelements(polylines)
print “OK.”
How to Become a Better MineSight® User
Page 8
MineSight® Grail—Data Access and Embedded Scripts
21st
Annual This script generates 10 polylines, a new MineSight® object within the datamanager, and
stores those polylines within that Datamanager object.
Seminar
Automating Triangulations
Running triangulates can be a time-consuming and repetitive task. Now with
MineSight® Grail v1.2 and Embedded Grail within MineSight® 3-D v3.4 you will be able to
run a triangulation routine on any Element that you extract from either the selection buffer
or from a Datamanager MineSight® Object (MOB).
The follow small script illustrates the triangulation of all data stored in MOBs within the
data folder in the Datamanager.
import sys
import os
import time
from grail import gsys
from grail import messages
from grail.ms3d import datamanager
from grail.ms3d import progressmeter
from grail.ms3d import mssys
from grail.ms3d import element
from grail.ms3d import elementop
sys.stdout = mssys.stdout # all “prints” go to the message window.
def gmain(msg, data):
“””Main entry point.”””
if msg is messages.gRUN:
run()
else:
return gsys.grailmain(msg, data)
def ispolyline(el):
“””Indicates if a given element is a polyline or not.”””
return el.gettype() in [element.PolylineType, element.PolygonType]
def run():
print “-”*50+”\nstarting triangulation (%s) ... “ % (sys.argv[0])
for item in datamanager.listfolder(“\\data”):
srcpath = os.path.join(datamanager.ROOT, “data”, item)
dstpath = os.path.join(datamanager.ROOT, “data”, “tri_”+item)
triangulateobj(dstpath, srcpath, “\\materials\\Geometry”)
print “ triangulated: ‘%s’ -> ‘%s’” % (srcpath, dstpath)
print “... done.\n”+”-”*50
def triangulateobj(dstpath, srcpath, materialpath):
mob = datamanager.open(srcpath)
polylines = filter(ispolyline, mob.getelements())
mob.close()
surface = elementop.triangulate(polylines)
mob = datamanager.create(dstpath,
datamanager.GEOMETRY_OBJECT,
materialpath)
mob.addelements([surface])
mob.close()
The above script works with the Datamanager to find polylines within the data folder to
triangulate. The triangulation in this case is a simple triangulation operation, but the effect
is a surface. Each surface is stored as “tri_xxx” where “xxx” is the original name of the
How to Become a Better MineSight® User
Page 9
MineSight® Grail—Data Access and Embedded Scripts
21st
Datamanager object that contained the polylines. Annual
Seminar
One can easily combine the triangulation operation shown above with the contouring
operation shown below. With these two steps one can take a series of polylines, generate a
surface, and then generate a contour interval to their specifications.
Automating Contouring
With the MineSight® Grail Embedded you will be capable of generating contours for a
given surface.
The contour operation is the basic creation of a polylines for a given elevation on a
surface. A smoothing operation is left as another operation to be executed on the resultant
contour polylines.
The following script will open every MineSight® geometry object in the \data directory
of the Datamanager. It will look for surfaces and generate contours for that surface. Those
contours will be stored back into a file prefixed with “cont”. Generally you would have
some specific set of regular and index contours you wish to generate for your surfaces,
but for purposes of illustration, this script will just apply a percent of contours base on the
elevation difference for a particular surface.
The contour.py script is as follows:
import os, sys
from grail import gsys, messages, ag
from grail.ms3d import datamanager, progressmeter,
from grail.ms3d import mssys, elementop, element
sys.stdout = mssys.stdout # redirect “print” to the message window
# Prefix in front of new geometries.
PREFIX = “cont_”
# Folder we scan for geometries to contour.
SAMPLE_FOLDER = “\\data”
# Decimal percent for the number of contours per surface.
CONTOUR_PERCENT = .50
def gmain(msg, data):
if msg is messages.gRUN:
runcontour()
else:
return gsys.grailmain(msg, data)
def runcontour():
materialpath = os.path.join(datamanager.ROOT,
“materials”,
“Geometry”)
for item in datamanager.listfolder(SAMPLE_FOLDER):
srcpath = os.path.join(datamanager.ROOT, “data”, item)
dstpath = os.path.join(datamanager.ROOT, “data”, PREFIX+item)
contouritem(dstpath, srcpath, materialpath)
print “ contoured: ‘%s’ -> ‘%s’” % (srcpath, dstpath)
def issurface(el):
return el.gettype() == element.ShellType
def contouritem(dstpath, srcpath, materialpath):
“””For all the surfaces in the srcpath, generate contours.”””
How to Become a Better MineSight® User
Page 10
MineSight® Grail—Data Access and Embedded Scripts
21st
Annual mob = datamanager.open(srcpath)
surfaces = filter(issurface, mob.getelements())
Seminar mob.close()
contours = []
for surface in surfaces:
min, max = ag.pointlistboundingvolume(surface.getpointlist())
minz, maxz = min[2], max[2]
diff = maxz-minz # only need to examine “Z”
step = diff/(diff*CONTOUR_PERCENT)
print “ contouring with: [%f, %f) and steps of %f” %\
(minz, maxz, step)
# append to our growing list of contours.
contours += elementop.contour(surface, minz,
maxz+ag.EPSILON, step)
# If we have contours from the above loop across the surfaces
# then generate a datamanager object and store the values.
if bool(contours):
mob = datamanager.create(dstpath,
datamanager.GEOMETRY_OBJECT,
materialpath)
mob.addelements(contours)
mob.close()
Future Plans
We have plans to enhance MineSight® Grail with the following:
• Provide direct access to the complete set of MineSight® Grail MSR files, including
drillhole views, model views, legends, etc.
• Enhance MineSight® Grail Embedded with more tools for automatic processing,
such as surface-surface intersections run from a MineSight® Grail script.
Provide a converter that will convert MineSight Compass procedures to MineSight®
Grail scripts .
Summary
Using Python with MineSight® Grail allows you to accomplish a multitude of tasks for
your business needs. The MineSight® Grail library will provide you with the ability to
access MineSight® data in any way that you see fit. In the future, MineSight® Grail will be
used to help automate and customize your MineSight® 3-D experience.
How to Become a Better MineSight® User
Page 11
