Skip to content

[NEW] Collect data for Monthly/Daily Active Users for a future dashboard#11525

Merged
sampaiodiego merged 39 commits intodevelopfrom
server-session-monitor
Jan 30, 2019
Merged

[NEW] Collect data for Monthly/Daily Active Users for a future dashboard#11525
sampaiodiego merged 39 commits intodevelopfrom
server-session-monitor

Conversation

@renatobecker-zz
Copy link
Copy Markdown

@renatobecker-zz renatobecker-zz commented Jul 20, 2018

New Server Session Monitor for SAU (Simultaneously Active Users), which will provide important informations for DAU (Active Daily Users) and MAU (Monthly Active Users).
The monitor is based on Meteor server sessions and will store the data in a new collection: rocketchat_sessions.

The approach of this implementation will deal with Connection events, such as onConnection and onClose, as well as dealing with Account events, such as onLogin and onLogout.

The SAU monitor will use the Meteor server sessions to update the session activities in DB. The monitor will ping the active sessions every minute, updating the lastActivityAt field on session documents.

The session lifecycle is stored per day, so when the server date is changed, the current sessions will be created on the new day. This behaviour is designed to facilitate the collector process, which will aggregate the session life cycle and store it in another collection of statistics.

The regular(non-mobile) session documents will look like the doc below:

{
	"_id" : "XSM2orqtSPWGYoQT2",
	"year" : 2018,
	"month" : 7,
	"day" : 20,
	"instanceId" : "LPvDxJh8fyDy6QsYz",
	"sessionId" : "ySTQpjSifhFfXWDis",
	"userId" : "uk294ojDzhYFQK53P",
	"ip" : "127.0.0.1",
	"host" : "localhost:3000",
	"browser" : {
		"name" : "Chrome",
		"version" : "67.0.3396.99",
		"major" : "67"
	},
	"os" : {
		"name" : "Mac OS",
		"version" : "10.13.6"
	},	
	"createdAt" : ISODate("2018-07-20T17:06:32.480Z"),  //when the session is created
	"lastActivityAt" : ISODate("2018-07-20T17:10:01.166Z"), //when the session monitor updates current sessions in bucket storage
	"loginAt" : ISODate("2018-07-20T17:06:32.491Z"), //user login
	"logoutAt" : ISODate("2018-07-20T17:10:07.311Z") //user logout
}

The mobile session documents will look like the doc below:

{
	"_id" : "YLoJ68zjA6WfNFt3b",
	"day" : 26,
	"instanceId" : "LPvDxJh8fyDy6QsYz",
	"month" : 7,
	"sessionId" : "wTrw5SNHa7wfudxaz",
	"year" : 2018,
	"ip" : "200.34.239.117,127.0.0.1",
	"host" : "d16877f6.ngrok.io",
	"os" : {
		"name" : "iOS",
		"version" : "11.4.1"
	},
	"device" : {
		"type" : "mobile"
	},
	"app" : {
		"name" : "RC Mobile",
		"version" : "v3.0.2",
		"bundle" : "(202)"
	},
	"createdAt" : ISODate("2018-07-26T18:57:49.475Z"),
	"userId" : "uk294ojDzhYFQK53P",
	"loginAt" : ISODate("2018-07-26T18:57:49.812Z"),
        "lastActivityAt" : ISODate("2018-07-26T18:58:03.834Z"),
	"closedAt" : ISODate("2018-07-26T18:58:03.834Z")
}

This new feature is being implemented in 3 steps:

  • Model design (sessions)
  • Monitor Implementation
  • Tests

This PR will close the issue number: #11461.

Once the implementation is approved, we need to think about the creation of a cron job that will collect the data every day, aggregating the data and storing in two other collections: SAU and MAU.

@renatobecker-zz renatobecker-zz requested a review from rodrigok July 20, 2018 18:29
@engelgabriel engelgabriel temporarily deployed to rocket-chat-pr-11525 July 20, 2018 18:29 Inactive
@engelgabriel engelgabriel temporarily deployed to rocket-chat-pr-11525 July 20, 2018 18:38 Inactive
@RocketChat RocketChat deleted a comment Jul 20, 2018
@engelgabriel engelgabriel temporarily deployed to rocket-chat-pr-11525 July 24, 2018 22:32 Inactive
@engelgabriel engelgabriel temporarily deployed to rocket-chat-pr-11525 July 25, 2018 14:47 Inactive
@engelgabriel engelgabriel temporarily deployed to rocket-chat-pr-11525 July 25, 2018 16:07 Inactive
@engelgabriel engelgabriel temporarily deployed to rocket-chat-pr-11525 July 27, 2018 14:21 Inactive
@renatobecker-zz renatobecker-zz changed the title [WIP][NEW]Server Session Monitor for SAU(Simultaneously Active Users) [NEW]Server Session Monitor for SAU(Simultaneously Active Users) Jul 27, 2018
@engelgabriel
Copy link
Copy Markdown
Member

Can we rename this packed to statistics? SAU is too specific and it is really just the begging of the final goal. We need a whole dashboard of several data poiont in the furutr.

@renatobecker-zz
Copy link
Copy Markdown
Author

@engelgabriel, Okay! I'm adding some improvements to this implementation, so I'm going to rename the package to be more generic, as you just suggested.

@rodrigok rodrigok temporarily deployed to rocket-chat-pr-11525 December 15, 2018 18:34 Inactive
@rodrigok rodrigok force-pushed the server-session-monitor branch from 3268b2f to 9544f6b Compare December 15, 2018 18:35
@rodrigok rodrigok temporarily deployed to rocket-chat-pr-11525 December 15, 2018 18:35 Inactive
@rodrigok rodrigok temporarily deployed to rocket-chat-pr-11525 December 15, 2018 18:44 Inactive
@rodrigok rodrigok temporarily deployed to rocket-chat-pr-11525 December 31, 2018 13:28 Inactive
@rodrigok rodrigok changed the title [NEW]Server Session Monitor for SAU(Simultaneously Active Users) [WIP][NEW] Server Session Monitor for SAU (Simultaneously Active Users) Jan 2, 2019
@rodrigok rodrigok temporarily deployed to rocket-chat-pr-11525 January 25, 2019 17:46 Inactive
@rodrigok rodrigok temporarily deployed to rocket-chat-pr-11525 January 27, 2019 00:00 Inactive
@sampaiodiego sampaiodiego temporarily deployed to rocket-chat-pr-11525 January 27, 2019 01:37 Inactive
@rodrigok rodrigok modified the milestones: Short-term, 0.74.0-rc.2 Jan 27, 2019
@rodrigok rodrigok temporarily deployed to rocket-chat-pr-11525 January 27, 2019 21:52 Inactive
Copy link
Copy Markdown
Member

@sampaiodiego sampaiodiego left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have found an issue that if an instance dies all their sessions will never receive a closedAt .. I'm wondering if that might break/distort aggregations

export class SAUMonitorClass {
constructor() {
this._serviceName = 'SAUMonitor';
this._started = false;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason to have this started state? I haven't found anything stopping it. and from what I saw there is nothing trying to start it multiple times as well.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a precaution I guess

@rodrigok rodrigok modified the milestones: 0.74.0-rc.2, 0.74.1 Jan 28, 2019
@rodrigok rodrigok changed the title [WIP][NEW] Server Session Monitor for SAU (Simultaneously Active Users) [NEW] Collect data for Monthly/Daily Active Users for a future dashboard Jan 29, 2019
@sampaiodiego sampaiodiego mentioned this pull request Feb 1, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants