The IBM Notes and Domino roadmap mantra *sigh*

I guess it was at IconUK Sept 2016 that IBM presented on their roadmap for Notes/Domino since a long time. The “big” news was that Q4 2016 IBM would come with an announcement regarding “updated tooling explored”.

What they meant with that was not clear to me, but “updated tooling” refers to me to what exists today, and that is currently just XPages if you ask me.

So last week during a webinar co-hosted by TeamStudio and TLCC the results of this exploration was expected to be presented as part of the roadmap for Notes and Domino. To my surprise instead of taking the opportunity and time to explain the findings of this exploration IBM simply repeats “updated tooling  explored” but adds the line “more during Connect 2017”. WHAT?!?

Didn’t you first come with a press notification that in Q4 2016 you will explain more about application modernization plans and invite people to come and listen to your views and then during this press report you simply say that people may wait another quarter?

And what does “more” in “more during Connect 2017” mean? How much more can people expect?

I am not sure what it means in English but it Dutch it sounds to me like a “zoethoudertje” but it has a negative interpretation. Probably the best translation is “compensation” It never satisfies any party in the long run.

Customers and developers want to move on and modernize their applications. And I am doubting if IBM is delivering it for Application Development in Notes Domino. If I look at the job market, the market is dead. If I read the modern development forums it is JavaScript (Node JS, Angular, React), Java, NonSQL databases where data is stored directly in JSON, GraphQL or any other Graph db, Websocket connection streams etcetera. Hell yeah blend in a couple of services from Bluemix if you like.

So yes, I am very disappointed in this announced “delay” and this unprofessional “procastination” by IBM. I am not sure what you are doing at your offices. Talk with your customers and development team. Write the technical opportunities down in a living document. Keep the document fresh and fruitful. Allocate sufficient resources so you come to deliverables.

Nobody wants to spend 2 hours listening to a roadmap-mantra that brings “nothing more on updated tooling explored” except Java 1.8, probably.

Development tip: learn some administration

I have no idea what your administration skills are. Mine date officially back to mid 2000 although I have setup recently servers but then mainly for development purposes only.

So if you want your skills and insights updated like I do or you  perhaps a tip for you could be to order the IBM Lotus Domino Administration 8.5.3 Training at Udemy.  Untill 1 November there is a give away action running to buy courses for 10 euro if you provide the coupon UDEMY10OCT.

Okay, it is targetted for 8.5.3 but how much has changed between 8.5.3 and 9.0.1?

I have no idea about the quality of the course, I guess you have to find out yourself.  For 10 euro: what are your alternatives?

Happy administration =)

Send a message from your XPage application to a Slack channel

Introduction

While everyone is ‘waiting in vain’ for the delivery of Project Toscana organizations have start using Slack as a modern way to send messages within departments.

For an example a operation team is using an instructions quality book build on XPages and it would be great to get notified in your Slack channel when new instructions come in or have been updated. Or you want to highlight a specific instruction just to create extra attention to it (e.g. an increasing number of incoming incidents).

slack_sample

Be creative with use-cases 🙂

Slack integration points

Incoming Webhooks are a simple way to post messages from external sources into Slack. They make use of normal HTTP requests with a JSON payload, which includes the message and a few other optional details described later.

JSON, HTTP request… this sounds doable with XPages! So what are the steps you must take?

Setup your integration

Go to http://yourteam.slack.com/apps/build/custom-integration and click on Incoming Webhooks, then select a channel or user you want to post your messages to.

Setup your message

You have two options for sending data to the Webhook URL above:

  • Send a JSON string as the payload parameter in a POST request
  • Send a JSON string as the body of a POST request

Simple message

For a simple message, your JSON payload could contain a text property at minimum. This is the text that will be posted to the channel e.g.:

payload={"text": "This is a line of text in a channel.\nAnd this is another line of text."}

You can decorate your message with text formatting. You will find descriptions in the custom integration page on Slack.

Message attachments

To display a richly-formatted message attachment in Slack, you can use the same JSON payload as above, but add in an attachments array. Each element of this array is a hash containing the following parameters:

{
	"fallback": "Required text summary of the attachment that is shown by clients that understand attachments but choose not to show them.",

	"text": "Optional text that should appear within the attachment",
	"pretext": "Optional text that should appear above the formatted data",

	"color": "#36a64f", // Can either be one of 'good', 'warning', 'danger', or any hex color code

	// Fields are displayed in a table on the message
	"fields": [
		{
			"title": "Required Field Title", // The title may not contain markup and will be escaped for you
			"value": "Text value of the field. May contain standard message markup and must be escaped as normal. May be multi-line.",
			"short": false // Optional flag indicating whether the `value` is short enough to be displayed side-by-side with other values
		}
	]
}

Send the message

With a simple post request you send the message in JavaScript:

function postMessageToSlack(url){
webhook_url = url;
payload= JSONStr;
var xmlhttp = new XMLHttpRequest();
xmlhttp.open('POST', webhook_url, false);
xmlhttp.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
xmlhttp.send(payload);
}

XSnippet

This whole approach I have added as an XSnippet on OpenNTF:

https://openntf.org/XSnippets.nsf/snippet.xsp?id=slack-integration

You can customize it to your needs. Basically I have a script block and a button on a custom control. Via the Property Definition tab you can set the desired values. In my example I am sending a message to a Slack channel about a picture / document in my Bildr application (shameless plug).

Here is how the message could look like:

sample_slack_message

Since my XPages app is running on a local development machine it is of no use to send sampel thumbnails of the page, but I could. This also refers to the user icon, which could be the icon of the application.

So grab the snippet and start integrating your XPages with Slack (untill Toscana arrives) !

FYI: I am not aware of any restrictions on the number of message you may send to Slack. Please let me know if you have details on this.

Add 20 years of experience to your workforce

You can 20 years of experience within IBM Notes and Web development to your workforce by hiring me.

Interested? Read my curriculum vitae on LinkedIn: http://www.linkedin.com/in/patrickkwinten and get in contact.

I am happy to work WITH you !

 

Update on my Cloudant sample XPages application

I made some additions to my XPages sample app that I am using to understand and demonstrate the usage of IBM Cloudant within XPages.

I am not fully there yet where I want to be, but it is a work in progress. During development I noticed a couple of things:

  • Cloudant’s Lite Pricing Plan has its limitations 🙂
  • Migrating Notes documents to Cloudant goes fast enough. A set of 10 thousands of simple, small documents takes just a couple of seconds.
  • Loading the same amount of documents into XPages e.g. a Repeat Control and navigate between sets via a Pager control has some delay time. This could be due the limitations of my local application server.

The code is available on Github.

cloudant_progress

Threads and Jobs on release 9

On OpenNTF you can find a project called Thread and Jobs:

This project contains samples showing how to create threads and Eclipse jobs from XPages to run longer taking operations asynchronously without blocking the XPages user interface or to run scheduled tasks.

It turns out that there have been updates in Release 9 in the ThreadSessionExecutor class and no longer a Status object is returned.

So if you want to have the example application working on your Domino 9 server you should remove the Status/iStatus library import and change the return type from Status to Object as followed:

/*
* © Copyright IBM Corp. 2012
*
* Licensed under the Apache License, Version 2.0 (the “License”);
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at:
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an “AS IS” BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing
* permissions and limitations under the License.
*/
package org.openntf.samples.thread;

import java.util.Date;
import com.ibm.domino.xsp.module.nsf.ThreadSessionExecutor;
import lotus.domino.Database;
import lotus.domino.Document;
import lotus.domino.NotesException;
import lotus.domino.Session;

/**
* Thread sample
*
* @author priand
* @author Niklas Heidloff (simplified sample)
*
* Updated 2016-10-18 Patrick Kwinten
*
*/

public class ThreadSample {

private static MyThread myThread;

public static boolean isRunning() {
return myThread != null;
}

public static void startThread() throws NotesException {
if (myThread != null) {
stopThread();
}
try {
synchronized(ThreadSample.class) {
if (myThread == null) {
myThread = new MyThread();
myThread.start();
}
System.out.println(“Thread started”);
}
} catch (Throwable t) {
t.printStackTrace();
}
}

public static void stopThread() {
if (myThread != null) {
synchronized(ThreadSample.class) {
if (myThread != null) {
myThread.stopRequest = true;
myThread = null;
System.out.println(” >> Thread stopping”);
}
}
}
}

private static class MyThread extends Thread {
boolean stopRequest;
private ThreadSessionExecutor < Object > executor;
MyThread() throws NotesException {
this.executor = new ThreadSessionExecutor < Object > () {
@Override
protected Object run(Session session) throws NotesException {
try {
System.out.println(” >> Thread running”);
Database db = session.getDatabase(null, “ThreadsJobs.nsf”);
if (db != null) {
if (!db.isOpen())
db.open();
if (db.isOpen()) {
System.out.println(” >> Database opened: ” + db.getTitle());
Document doc = db.createDocument();
try {
doc.replaceItemValue(“Form”, “ThreadTest”);
doc.replaceItemValue(“DateTime”, session.createDateTime(new Date()));
doc.save();
} finally {
doc.recycle();
}
}
}
} catch (Throwable ex) {
ex.printStackTrace();
}
if (!stopRequest) {
try {
Thread.sleep(5 * 1000);
} catch (InterruptedException ex) {
ex.printStackTrace();
}
}
//return Status.OK_STATUS;
return “probably OK”;
}
};
}

public void run() {
while (!stopRequest) {
try {
executor.run();
} catch (Exception ex) {}
}
System.out.println(“Thread left”);
}
}
}

I have not yet managed how to fix the JobScheduler class…

MapReduce views

Introduction

Frank van der Linden posted he made an update to his Cloudant connector plugin, so from now it should be possible to manage design documents.

To become more familiar with couchDB’s design documents I can recommend reading ‘Writing and Querying MapReduce Views in CouchDB‘.

Design documents

Design documents you mostly manage in CouchDB / Cloudant are documents that describe views of documents. Great if you can manage them via http and not as in IBM Notes via the Domino Designer client 😉

You could store other documents in your couchDb which are part of the design of your application (e.g. html, css, js files but that is another story).

Design documents are nothing more than JSON documents which are stored under the convention: must start with the _design/ pattern followed by the identifier e.g.: _design/default.

MapReduce

Views in couchdb are based on the MapReduce principle:

A MapReduce program is composed of a Map() procedure that performs filtering and sorting (such as sorting students by first name into queues, one queue for each name) and a Reduce() procedure that performs a summary operation (such as counting the number of students in each queue, yielding name frequencies).

This video explains it by using playing cards.

Mapping

The filtering and sorting are defined by a JavaScript function. You can state conditions in the function to return the documents you are interested in e.g.:

function(doc) {
if (doc.labels) {
for (var idx in doc.labels) {
if (doc.labels[idx] === “organic”) {
emit(doc._id, doc.name);
}
}
}
}

//docs have to have the field doc.labels and only the id and name from the ones that have the label organic are returned

Reducing

Reducing is optional and results in a summary operation. The options in couchDb are:

  • _sum
  • _count
  • _stats

Count is probably the most common function to use. Notice that the returned key value might be null if values appear in multiple documents. The key is no longer unique and therefor null.

{
“rows”: [{
“key”: null,
“value”: 7
}]
}

If you apply the group parameter in the requested url the values will be grouped so the response could look like:

{
“rows”: [{
“key”: “appel”,
“value”: 3
}, {
“key”: “banaan”,
“value”: 2
}, {
“key”: “peer”,
“value”: 2
}
]
}

Sum works only on numeric values (of course) so if your mapping functions looks like:

function(doc) {
if (doc.fruits) {
for (var i in doc.fruits) {
emit(doc.fruits[i], doc.calories);
}
}
}

with the _sum reduce option the response could look like:

{
“rows”: [{
“key”: “appel”,
“value”: 720
}, {
“key”: “banaan”,
“value”: 1368
}, {
“key”: “peer”,
“value”: 320
}]
}

The _stats option returns a JSON object containing the sum, count, minimum, maximum, and sum over all square roots of mapped values. So for the mapping function above the response could be:

{
“rows”: [{
“key”: “appel”,
“value”: {
“sum”: 720,
“count”: 2,
“min”: 272,
“max”: 448,
“sumsqr”: 274688
}
}, {
“key”: “banaan”,
“value”: {
“sum”: 1368,
“count”: 3,
“min”: 272,
“max”: 648,
“sumsqr”: 694592
}
}, {
“key”: “peer”,
“value”: {
“sum”: 720,
“count”: 2,
“min”: 272,
“max”: 448,
“sumsqr”: 274688
}
}]
}

With this knowledge fresh in mind you can use them in your view(s) setup. Remember the design documents can be managed via CRUD operations, e.g. on address http://localhost:5984/veganstore/_design/default

{
“_id”: “_design/default”,
“language”: “javascript”,
“views”: {
“store”: {
“map”: “function(doc) { if (doc.label) { emit(doc.label, doc.origin); }}”,
“reduce”: “_stats”
}
}
}

In one design document you can define multiple views…

You can query your views by a set of parameters:

  • conflicts (boolean) – Includes conflicts information in response. Ignored if include_docs isn’t true. Default is false
  • descending (boolean) – Return the documents in descending by key order. Default is false
  • endkey (json) – Stop returning records when the specified key is reached. Optional
  • end_key (json) – Alias for endkey param
  • endkey_docid (string) – Stop returning records when the specified document ID is reached. Requires endkey to be specified for this to have any effect. Optional
  • end_key_doc_id (string) – Alias for endkey_docid param
  • group (boolean) – Group the results using the reduce function to a group or single row. Default is false
  • group_level (number) – Specify the group level to be used. Optional
  • include_docs (boolean) – Include the associated document with each row. Default is false.
  • attachments (boolean) – Include the Base64-encoded content of attachments in the documents that are included if include_docs is true. Ignored if include_docs isn’t true. Default is false.
  • att_encoding_info (boolean) – Include encoding information in attachment stubs if include_docs is true and the particular attachment is compressed. Ignored if include_docs isn’t true. Default isfalse.
  • inclusive_end (boolean) – Specifies whether the specified end key should be included in the result. Default is true
  • key (json) – Return only documents that match the specified key. Optional
  • keys (json-array) – Return only documents where the key matches one of the keys specified in the array. Optional
  • limit (number) – Limit the number of the returned documents to the specified number. Optional
  • reduce (boolean) – Use the reduction function. Default is true
  • skip (number) – Skip this number of records before starting to return the results. Default is 0
  • sorted (boolean) – Sort returned rows (see Sorting Returned Rows). Setting this to false offers a performance boost. The total_rows and offset fields are not available when this is set to false. Default is true
  • stale (string) – Allow the results from a stale view to be used. Supported values: ok and update_after. Optional
  • startkey (json) – Return records starting with the specified key. Optional
  • start_key (json) – Alias for startkey param
  • startkey_docid (string) – Return records starting with the specified document ID. Requires startkey to be specified for this to have any effect. Optional
  • start_key_doc_id (string) – Alias for startkey_docid param
  • update_seq (boolean) – Response includes an update_seq value indicating which sequence id of the database the view reflects. Default is false

With this summary on views in couchDBcloudant fresh in mind let’s try to create some views programmatically in Cloudant with Frank’s updated plugin =)

Using IBM Cloudant with XPages

Last evening I started a little couch development project, in front of the TV. The data I am used to work with is mostly in JSON format already so I read a bit on couchDB. Since my use-case allows storage in the cloud I was curious about Frank van der Linden’s cloudant-connector plugin so I gave it a swing.

Cloudant is the cloud version of CouchDb. Which is invented by Damien Katz, who also was involved by the creation of the NSF.

I found on the internet a very nice easy to use open source project, Java-cloudant, who is doing the heavy lifting in the interaction with Cloudant.

I guess for an XPages developer this is the easiest way to get started with Cloudant. You can also load the libraries yourself as Stephan Wissel describes. At this point I am not sure what option I like best.

It turns out with Frank’s plugin you can develop really fast rapid. If you look at the HR-assistant application he has worked on you find some great examples how to use it. Nevertheless that project was too sophisticated for me to adapt in my 2hr sample app. My goal was to learn more about couchDb and IBM Cloudant and working with documents & views in general and not (yet) building applications around it.

So this is the result I achieved in my restricted time:

cloudant-sample

Nothing much sophisticated, but a good learning experience which I would like to take with me in my next “couch” project 🙂 The code is available in a Github repo.

In a month Frank is holding a presentation at SUTOL concerning using developing with Cloudant so I am looking forward to hear what he has to say and show there 🙂

Recently(?) IBM Cloudant on Bluemix has a “Lite” pricing plan which is free with restrictions so it’s developers friendly:

cloudant-plan