1.
#engageug
De13. How to use the new
Domino Query Language
Tim Davis, The Turtle Partnership Ltd
1

2.
#engageug
Domino Query Language
• New with Domino 10
• Sophisticated search facility that can make use of design
elements (e.g. views)
• Uses pre-harvested design data, and breaks down query
terms to run queries efficiently.
• Returns a collection of documents
• Can use it in LotusScript, Java, Node.js
2

3.
#engageug
Requirements
• Domino 10+ (current latest version is 10.0.1)
• To run DQL queries you need a design catalog on the
server
• GQFdsgn.cat
• Contains the special indexing
• Eventually this indexing will be stored inside the actual
databases
3

4.
#engageug
Setup
• Create the design catalog
• You need at least one database indexed to use DQL
• You do this using updall with the new -e option
load updall mydb.nsf -e
• This creates the GQFdsgn.cat with the indexing for this
database
• In theory you can query other databases, but it will be
slow and may timeout
4

5.
#engageug
Updating
• If you modify the design of the database, you will need to
update the design catalog
load updall mydb.nsf -d
• Otherwise the catalog won't know about the changes
• There isn't an automatic update process yet
5

6.
#engageug
Limitations
• Not intended for users to build queries on the fly
• No full text or attachments yet
• Only runs on server (needs the design catalog)
• Can't index secure views
• Don't do massive searches, there are configurable limits on
queries to protect server
• Tune the queries for performance, use 'explain'
6

7.
#engageug
Limits
• There are built-in limits to prevent huge queries affecting the
server
• MaxDocsScanned
• maximum allowable NSF documents scanned ( 200000
default)
• MaxEntriesScanned
• maximum allowable index entries scanned (200000
default)
• MaxMsecs
• maximum time consumed in milliseconds ( 120000, 2
minutes)
7

8.
#engageug
Overriding limits
• Notes.ini settings (server-wide):
• QUERY_MAX_DOCS_SCANNED
• QUERY_MAX_VIEW_ENTRIES_SCANNED
• QUERY_MAX_MSECS_TOTAL
• Can also set these for specific queries in LS, Java, and
Node
8

9.
#engageug
Basic Syntax
• Field value: fieldname = 'value'
• View column: 'viewName'.columnProgName
• Usual operators: = > < >= <=
• Booleans: and, or, and not, or not
• Parentheses for precedence ( )
form = 'Person' and lastname = 'Baker' and
'Personview'.dept = 'Sales'
9

10.
#engageug
More Syntax
• Strings are single quotes: 'Bob'
• Escape single quotes with two single quotes
• e.g. "It's Susan's cat" is escaped as
"It''s Susan''s cat"
• Numbers are floating point
• Can be without decimal point (integers)
• Can be scientific notation
10

11.
#engageug
Even More Syntax
• Dates use the @dt() function, in RFC3339 format
• dateSubmitted > @dt('2019-04-25T09:00:00+0500')
• You can have partial date or time strings
• @dt('2019-04-25')
• @dt('09:00:00')
• These don't work in view column queries, use whole
string
• Dates are UTC/GMT, unless you specify a timezone with a
+/- modifier
11

12.
#engageug
Too Much Syntax
• There are some built-in @functions
• @ModifiedInThisFile
• @Created
• @DocumentUniqueID
• @All
12

13.
#engageug
Bit More Syntax
• Lists
• Use the 'in' operator
• Can be value lists or view lists
• city in ('London', 'New York', 'Paris', 'Munich')
• in ('PendingOrders', 'DeliveredOrders', 'InvoicedOrders')
• Use 'in all' to match all items, rather than any
13

14.
#engageug
Bit More Syntax
• Lists
• Use the 'in' operator
• Can be value lists or view lists
• city in ('London', 'New York', 'Paris', 'Munich')
• in ('PendingOrders', 'DeliveredOrders', 'InvoicedOrders')
• Use 'in all' to match all items, rather than any
• fieldname in (list) means query as values
• in (list) on its own means query as views
14

15.
#engageug
Example queries
"lastname = 'Crawford'"
"lastname = 'Crawford' and city = 'Plano'"
"'AllContacts'.LastName = 'Henson'"
"lastname in ('Flynn', 'Foley', 'Fowler')"
"form = 'Contact' and @Created > @dt('2018-05-10')"
"lastname = 'Crawford' and in ('AllContacts', 'AllThreads')"
15

16.
#engageug
Performance
• Terms that run in harvested view scans are fast
• Views are automatically used for field=value terms if they
are 'collated'
• @All, with sorted column with that field as value
• Create one to speed up your queries
• Can pre-populate folders to make queries within that folder
fast
• Use 'explain' to time how each part of the query runs
16

17.
#engageug
Explain
form = 'Contact' and @Created > @dt('2018-05-10')
0. AND (childct 2) (totals when complete:)
Prep 0.0 msecs, Exec 6.964 msecs, ScannedDocs 981, Entries 0, FoundDocs 2
1.form = 'Contact' NSF document search estimated cost = 100
Prep 0.118 msecs, Exec 6.959 msecs, ScannedDocs 981, Entries 0, FoundDocs 202
1.@Created > 2018-05-10T-1:-1:-1.-1Z NSF document search estimated cost = 100
Prep 0.79 msecs, Exec 0.0 msecs, ScannedDocs 202, Entries 0, FoundDocs 2
17

18.
#engageug
Explain explained
form = 'Contact' and @Created > @dt('2018-05-10')
0. AND (childct 2) (totals when complete:)
Prep 0.0 msecs, Exec 6.964 msecs, ScannedDocs 981, Entries 0, FoundDocs 2
1.form = 'Contact' NSF document search estimated cost = 100
Prep 0.118 msecs, Exec 6.959 msecs, ScannedDocs 981, Entries 0, FoundDocs 202
1.@Created > 2018-05-10T-1:-1:-1.-1Z NSF document search estimated cost = 100
Prep 0.79 msecs, Exec 0.0 msecs, ScannedDocs 202, Entries 0, FoundDocs 2
• Used nsf scan (not a collated view), counts scanned docs and found
docs
• Estimated cost is relative between terms (0-100)
• Applies more expensive terms to already filtered results 18

19.
#engageug
Useful tools
• Domquery
• Command to test queries
• load domquery -f filepath -q "<query>"
• Use its -e option to get explain output
• DQL Explorer
• Browser tool to run DQL queries
• https://www.ibm.com/blogs/collaboration-solutions/
2019/03/12/dql-explorer-is-now-available-on-github-and-
openntf/
19

20.
#engageug
LotusScript
• New query class and new methods for existing classes
• NotesDominoQuery
• query = db.CreateDominoQuery()
• query.execute( query$ )
• Returns a document collection
• Just like a search
• There are Explain() and Parse() methods to help
20

21.
#engageug
LS Example
Dim session As New NotesSession
Dim db As NotesDatabase
Set db = session.currentDatabase
Dim coll As NotesDocumentCollection
Dim dq As NotesDominoQuery
Set dq = db.createDominoQuery()
Set coll = dq.execute("LastName = 'Crawford'")
21

22.
#engageug
LotusScript Demo
22

23.
#engageug
Explain output examples
23

24.
#engageug
Java
• Similar to LotusScript
• New query class and methods
• DominoQuery
• DominoQuery dq = db.createDominoQuery();
• dq.execute(query);
• You have the explain(), parse() methods here, too
24

25.
#engageug
Java Example
Session session = getSession();
Database db = session.getCurrentDatabase();
DominoQuery dq = db.createDominoQuery();
DocumentCollection coll = dq.execute("LastName = 'Crawford'");
25

26.
#engageug
Java Demo
26

27.
#engageug
Node.js
• New with Domino 10, ability to access from node.js apps
• New 'domino-db' module for node
• New Proton task on the server
• Uses DQL to find domino documents
• Results are javascript objects and arrays
• My intro to node: https://www.slideshare.net/timdavis2k/
what-is-nodejs-108951618
27

28.
#engageug
Requirements
• Node.js v10
• Need the App Dev pack
• Check the online documentation:
• https://doc.cwpcollaboration.com/appdevpack/docs/en/
homepage.html
28

29.
#engageug
Setup
• Admin installs the Proton add-in on the server
• Proton task must be running.
• You may need a certificate and credentials if Proton is
using TLS/SSL
• Get the domino-db module installer archive:
• domino-domino-db-1.1.0.tgz
• (In future on npm, but currently a local install)
29

30.
#engageug
Install domino-db
• We install domino-db into our node app
• Put the module install tgz archive in a folder, e.g. packages/
• From the node app project folder, in command window or
terminal, run the install:
npm install /packages/domino-domino-
db-1.1.0.tgz --save
• This gets the code and updates app dependencies
30

31.
#engageug
Using the domino-db module
• 'Require' the domino-db module and create a server
connection
const { useServer } = require('@domino/domino-db');
• Connect to the server and get a server object
useServer(serverConfig).then(async server => {})
• Get the database
const database = await server.useDatabase(databaseConfig);
31

32.
#engageug
Using DQL in Node
• Get some documents with a 'bulk' read
const resp = await database.bulkReadDocuments({...});
• This does a DQL query
query: "LastName = 'Crawford'",
• Specify which fields you want returned
itemNames: ['FirstName', 'LastName']
• (By default only returns metadata, e.g. unid)
32

33.
#engageug
Node Example (pt 1)
const { useServer } = require('@domino/domino-db');
const serverConfig = {
hostName: 'server.domain.com', // Host name of your server
connection: {
port: '3002', // Proton port on your server
},
};
const databaseConfig = {
filePath: 'node-demo.nsf', // The database file path
};
33

34.
#engageug
Node Example (pt 2)
useServer(serverConfig).then(async server => {
const database = await server.useDatabase(databaseConfig);
const query = "LastName = 'Crawford'";
const coll = await database.bulkReadDocuments({
query: query,
itemNames: ['FirstName', 'LastName']
});
}).catch(error => {
console.log(error);
});
34

35.
#engageug
Results
• Javascript objects and arrays
• (all the details are in the online documentation)
• The result is a 'bulkResponse' object
• This has a documents property, which is an array of
document objects
• By default each only has @unid, @created, @modified
• Includes field values if you specified them
35

36.
#engageug
Using results
• In our example, resp.documents is an array of documents
• Can step through it and get named values in each
document
resp.documents[i].LastName
resp.documents[i]['LastName']
• Item types include 'text', 'datetime', 'number', 'reference'
• Can also be arrays, or pairs, or arrays of pairs
36

37.
#engageug
Node demo
37

38.
#engageug
Substitution values
• When running any query (SQL, DQL, etc), need to careful
of code injection
• Values in the query are actually code which can do stuff
• Vulnerable if building queries as simple strings
• Avoid this with 'query arguments', where the server inserts
query values into the query in a controlled way
• Ensures the arguments are treated only as values so
they cannot be interpreted as code
38

39.
#engageug
Query Arguments
• Use ? notation in the query to indicate named arguments
"orderNo = ?ordno and custNo = ?cno"
• LotusScript and Java define values like this:
dq.setNamedVariable("ordno", orderNo)
• Use resetNamedVariables() to clear these between
queries
• In node.js, use 'queryArgs' option in bulkReadDocuments
(alongside the query and itemValues options)
queryArgs: [{ name: 'ordno', value: orderNo }]
39

40.
#engageug
Futures
• Rich text items (v11)
• 'Contains' operator (v11)
• Catalog inside database nsf, no catalog db (v11)
• Cataloguing secured/reader views (v11)
• Reporting of DQL status
• Multiple databases
• @formulas
40

41.
#engageug
• My articles and presentations:
https://turtleblog.info/tag/node-js/
• DQL documentation:
https://doc.cwpcollaboration.com/appdevpack/docs/en/
domino-query-language.html
41