Monitoring Desktop Sync

There are a number of areas that can be monitored, including:

• the ActiveMQ topic that is used to relay events from the repository to the synchronization service.
• the health of the synchronization service.

ActiveMQ provides a publish/subscribe mechanism to relay node events from the repository to the synchronization service. The JMX beans that exposes information on the node events topic is:

org.apache.activemq:type=Broker,brokerName=localhost,destinationType=Topic,
destinationName=VirtualTopic.alfresco.repo.events.nodes

The synchronization service consumes from a queue that is tied to the virtual topic. The JMX bean name of this queue is:

org.apache.activemq:type=Broker,brokerName=localhost,destinationType=Queue,
destinationName=Consumer.<SyncId>.VirtualTopic.alfresco.repo.events.nodes

Advisories can be read like any other topic. The following advisories can be useful to monitor:

• sendAdvisoryIfNoConsumers (For the node events, this advisory is not sent because events are persistent)
• advisoryForSlowConsumers
• advisdoryForFastProducers
• advisoryForDiscardingMessages
• advisoryWhenFull

The advisories can be configured in the activemq.xml file as follows:

<policyEntry topic="VirtualTopic.alfresco.repo.events.nodes" advisoryForDelivery="true" 
advisoryForConsumed="true" advisoryForSlowConsumers="true" sendAdvisoryIfNoConsumers="true" 
advisoryForFastProducers="true">

The synchronization service health check can be accessed using JMX (bean health) or a REST call:

GET https://localhost:9090/alfresco/healthcheck

The output is something like:

{
  "versionCheck" : {
    "healthy" : true,
    "message" : "    1.1-SNAPSHOT (2016-03-09T11:12:52Z)"
 },
   "eventsHealthCheck" : {
    "healthy" : true,
    "message" : "Ok"
 },
   "activeMQConnection" : {
    "healthy" : true,
    "message" : "ActiveMQ connection Ok"
 },      
   "deadlocks" : {
    "healthy" : true
 },
   "repositoryConnection" : {        
    "healthy" : true,        
    "message" : "Repository connection Ok"
 },
   "databaseConnection" : {
    "healthy" : true,
    "message" : "Database connection Ok"
 },
   "syncServiceIdCheck" : {
    "healthy" : true,
    "message" : "00f51927-12d5-4f1b-9979-287fa6fe2771"
 }
}

Elements	Description
"eventsHealthCheck" : {    "healthy" : true,    "message" : "Ok" },	This specifies if the health check is up to date. If the status is unknown, it could be that there haven't been any events since the last restart. If the sync service falls behind by 10 seconds or more, the healthy element of the health check would say false.
"activeMQConnection" : { "healthy" : true,    "message" : "ActiveMQ connection Ok" },	This specifies that the connection to ActiveMQ is healthy.
"repositoryConnection" : {         "healthy" : true,         "message" : "Repository connection Ok" },	This specifies that the synchronization service can connect to SkyVault, for example, hostname and port in config.yml.
"databaseConnection" : { "healthy" : true,    "message" : "Database connection Ok" },	This specifies that the synchronization service is connected to the database and making successful SQL calls.
"syncServiceIdCheck" : {    "healthy" : true,    "message" : "00f51927-12d5-4f1b-9979-287fa6fe2771" }	This specifies the ID used in the connection to ActiveMQ to identify itself to the queue. See the

The config.yml file contains the logging information.

logging:
  level: INFO
  loggers:
    "org.alfresco.service.common.auth": WARN
    "org.apache.activemq": WARN
    "com.sun.jersey.api.container.filter.LoggingFilter": WARN
    "org.alfresco.service": INFO
  appenders:
    - type: console
      threshold: ALL
      timeZone: UTC
      target: stdout
      logFormat: "%-5level [%d{yyyy-MM-dd HH:mm:ss.SSS}] [%thread] %logger - %msg%n"
    - type: file
      threshold: ALL
      timeZone: UTC
      currentLogFilename: ./logs/sync-service.log
      archive: true
      archivedLogFilenamePattern: ./logs/sync-service-%d.log.gz
      archivedFileCount: 5
      logFormat: "%-5level [%d{yyyy-MM-dd HH:mm:ss.SSS}] [%thread] %logger - %msg%n"

The synchronization service exposes a collection of metrics that are useful in managing the synchronization service. The synchronization service metrics can be accessed using JMX (see bean metrics) or a REST call:

GET https://localhost:9090/alfresco/api/-default-/private/alfresco/versions/1/metrics

The response is JSON and contains all the metrics collected by the synchronization service. In particular:

Metric Name	Type	Description
nodeEventLag	Timer*	Specifies the time taken for events sent by the repository to be consumed by the synchronization service. It is measured in milliseconds.
nodeEventsBrokerTime	Timer	Specifies the amount of time the event spends in the (ActiveMQ) broker.
nodeEventBrokerLag	Timer	Specifies the lag between events being sent to the (ActiveMQ) broker by the repository and the time at which the broker receives the event.
nodeEventConsumerLag	Timer	Specifies the lag between the broker sending out events and the synchronization service consuming them.
lagBetweenEventCreateAndSend	Timer	Specifies the lag between the event creation time and the time the event is sent to the (ActiveMQ) broker.
syncsTimedOut	Meter**	Specifies the syncs that have timed out (possibly due to long query times).
syncFailuresMeter	Meter	Specifies the sync failures.
syncsTimer	Timer	Specifies the distribution of sync times.
timePerCommit	Timer	Specifies the node event database commit times.
timePerEventUpdate	Timer	Specifies the node event database insert times.
timePerGetChanges	Timer	Specifies the sync query times.