The Big Picture

Rocket.Chat already offers options to integrate external tools and systems on multiple levels:

• Incoming and outgoing webhooks allow sending (http-) messages back and forth to an external system
• For some external services, a native implementation of the other system's API is even shipped along with Rocket.Chat. e. g. for Zapier, you don't even need to know the API, you simply configure it.

However, what's missing is a comfortable, update-safe option to create custom applications living inside the Rocket.Chat Main application which bring along their own GUI and can be shipped (and maintained) for multiple Rocket.Chat instances.

At DB, we do have such a usecase: We integrate an NLP engine analyzing a conversation in order to speed-up communication within the room. We implemented this (painfully) as a modification of Rocket.Chat. Though there are quite some APIs which reduce the number of necessary places for modifications, each release of Rocket.Chat urges us to merge and check for merge-conflicts. Initially this issue was about providing a more flexible integration of AI-providers, but discussing this issue it turned out that actually the necessary achitecture is not limited to AI: It affects all cases where a microservice-like application shall be integrated into Rocket.Chat

Thus, dedicated Plugins-APIs which allow for modification-free enhancements which can be shipped along with Rocket.Chat is what this issue is about.

Use case

Rocket.Chat comes with a sample AI-integration for api.ai.
Natural-language-integration into a communication is a feature which is not only hyped, but can significantly speed-up-communication.
However, there are plenty of other AI-provides. This requirement aims at making Rocket.Chat ready for an easy integration with other "AI-providers".

Features

A Registry shall provide an API for connection of other AI-adapters.

• On the server, it basically allows to implement an interface with the following features:
  • initialize(): is called on startup and after a change of the configuration
  • onMessage(message): Allows triggering of an asynchronous request for analysis
  • onClose(room): Allows AI to learn from the conversation
  • onMessageUpdated(message): Allows AI to invalidate results
  • getSettings(): creates settings in the AI group and observes them for changes
• Client-side, there are basically two options how results of the AI can be visualized: Either using a mashup-mechanism (some globally registered script which is attached to a DOM node) or using a visualization built-into Rocket.Chat (Meteor).
  • onStartup(): Hook for loading an external script
  • setTabbar(): If a custom Blaze-template shall visualize the results, this template shall be loaded to the tabbar. The room and the data retrieved by the plugin from the external system shall be passed to the template.

Ideas for updating the external services' results

(in our case the NLP analysis)

The tricky part (such as I can see) is to actually make the UI update. This challenge does not only apply to the AI-UI, but to all third party applications which shall live inside the tabbar. Basically there are two options:

• The integrated UI opens an own websocket connection to its very own backend

• +Very clear separation of concerns
• -Overall resource consumption
• -The client's websocket limit might be exceeded

• Callback mechanism (Webhook): The API for registering the (AI) provider creates a webhook which can be passed to the foreign system.

• -Overhead at the RC server
• +Resource consumption: Re-use of the existing websocket connection
• +A server-side collection-based cache can be provided generically -> performance

Why not use an outgoing integration?

what's the difference for a full-stack-integrated application

• An external application might be stateful
• Multiple integrations (one for each callback event) were needed => lack of reuse
• Comfort of shipping a package > shipping multiple integrations
• High volume - custom logging might be useful
• Visualization is necessary (potentially based on other frameworks than Meteor)
• There is no option to ship more complex integrations-configuration