I am very interested in Graph data modelling and with the Graph capabilities in OpenNTF Domino API I decided to setup some demo environments just to get my head around the subject and how you can implement it in XPages and use Notes data.
To my opinion reading Notes data in the Graph database structure can bring interesting new opportunities, far beyond what we can deliver with Views and Collections today.
Oliver Busse has provided a great starting point with his SUTOL demo application so I started with that one.
Besides the implementation and Graph capabilities I am also curious about performance. So I run some tests on my working demo app. With the help with a simple agent I decided to be gentle and create a set of only 20.000 user documents.
The first test was about returning user profiles (nodes) matching certain properties (relations), presented in a repeat control. Below the list the time to load the filtered set is displayed.
When I compare the result with a normal view filter (by category or FT search) the results where a bit disappointing.
I also noted that navigating through the list was very slow (20 seconds or more returning a new set of rows of 10 documents). More than I expected I received timeouts.
The reason for this performance is still unknown. I guess there is no index created yet for the user node in the graph db structure. Why navigating through the list is so latent in performance is also a mysterie.
Nevertheless, my demo is up and running so expect more results on Graph in XPages with Notes data in the future on this blog.
Below are sampels of performance using the “traditional” FT search filter capacity in Notes. Notice the difference.
I would like to thank Oliver Busse for his guidance getting the demo app up and running and for explaining some basic concepts of the implementation.
That’s rather surprising and disappointing. Have you tried to see what’s going on on the server using the Xpages toolbox?
if I check the toolbox I see some remarkable differences. a bit hard to paste in here tabular data.
for the normal FT Search:
Class Method Operation Calls Time
View GetAllEntriesByKey 1 290
Database FTSearch 1 120
Document UniversalID Get 30 10
Document GetItemValue 40 0
DocumentCollection GetNextDocument 9 0
Session EffectiveUserName Get 6 0
Session CreateDateTime 3 0
Session CurrentDatabase Get 2 0
Database QueryAccessRoles 2 0
Database IsOpen Get 1 0
Database ReplicaID Get 1 0
Database GetDocumentByUNID 1 0
Database QueryAccess 1 0
Database GetView 1 0
Session GetDatabase 1 0
Session GetDbDirectory 1 0
Session International Get 1 0
DbDirectory Name Get 1 0
DbDirectory GetClusterName 1 0
DocumentCollection Count Get 1 0
DocumentCollection GetFirstDocument 1 0
International IsTime24Hour Get 1 0
International IsDateMDY Get 1 0
International IsDateDMY Get 1 0
International IsDateYMD Get 1 0
International AMString Get 1 0
International PMString Get 1 0
International DateSep Get 1 0
International TimeSep Get 1 0
Session AgentContext Get 1 0
ViewEntryCollection Count Get 1 0
DateTime TimeDifferenceDouble 1 0
Database QueryAccessPrivileges 1 0
DbDirectory recycle 1 0
International recycle 1 0
For the graph variant:
Class Method Operation Calls Time
Database GetDocumentByUNID 8203 822
Database OpenByReplicaID 4096 742
Session Evaluate 4106 321
Document HasItem 45299 240
View GetAllEntriesByKey 1 231
Document NoteID Get 8253 140
Document GetItemValue 14409 90
NoteCollection BuildCollection 1 90
Database Server Get 4097 40
Database FilePath Get 4097 30
Document GetFirstItem 4148 30
Database ReplicaID Get 4098 20
Item Name Get 8294 20
Document UniversalID Get 8213 20
Session GetDatabase 4099 20
DbDirectory Name Get 4099 10
DbDirectory GetClusterName 4099 10
Document recycle 4904 0
Database IsOpen Get 4107 0
Session GetDbDirectory 4099 0
DbDirectory recycle 4099 0
Session AgentContext Get 4098 0
Database recycle 4097 0
Item recycle 3626 0
NoteCollection GetUNID 10 0
Session EffectiveUserName Get 3 0
Session CreateDateTime 3 0
Session CurrentDatabase Get 2 0
Document GetItemValueCustomDataBytes 2 0
Database Open 1 0
Database GetView 1 0
Item ValueLength Get 1 0
Session International Get 1 0
International IsTime24Hour Get 1 0
International IsDateMDY Get 1 0
International IsDateDMY Get 1 0
International IsDateYMD Get 1 0
International AMString Get 1 0
International PMString Get 1 0
International DateSep Get 1 0
International TimeSep Get 1 0
ViewEntryCollection Count Get 1 0
DateTime TimeDifferenceDouble 1 0
NoteCollection SelectDocuments Set 1 0
NoteCollection SelectionFormula Set 1 0
Database CreateNoteCollection 1 0
NoteCollection Count Get 1 0
NoteCollection GetNoteIds 1 0
International recycle 1 0
NoteCollection recycle 1 0
Check out DFramedTransactionalGraph.getFilteredElements() and getFilteredElementsPartial(). These return either a FramedVertexList or a FramedEdgeList, both of which support .sortBy() and .subList()
That should push your performance level up quite a bit. I constantly work with graphs with upward of 100K nodes and can get results in <100 ms.
then I definitely doing something wrong
Can you share your demo code some place where I can pull it down and tweak it?