Investigating Provisioning Stuck Messages in RabbitMQ

Investigating Provisioning Stuck Messages in RabbitMQ

Overview

This is a working document for investigating stuck messages in RabbitMQ queue.

Key points

Unacked Messages are "unacknowledged messages". It's a message that is sent to but not successfully completed and just got stuck in the processing.

Stuck Message (Sample)

RabbitMQ Message is stuck in unacked

Steps for Investigation and Releasing of Unacked Message

1. Stop the AtriaProvisioningService. This will put back the stuck message on "Ready" status
2. Run SQL Profiler on the Database Server, this will help us see what will be processed
3. Start the AtriaProvisioningService. This will then try to re-run the message that got stuck.
4. Review the SQL Profiler tracing to see what is the action it was trying to do. It might give a clue on the reason why it's not being processed. Take note of it as we will skip the stuck message on the next step.
5. Again, stop the AtriaProvisioningService.
6. In RabbitMQ, open the queue that has a stuck message
7. Set the Ack Mode to "Automatic ack"; Messages: "1"

8. Click on Get Message(s). This will automatically acknowledge the 1st message on the queue which is the one getting stuck.
9. Start AtriaProvisioningService. This will then continue to process the other messages.


Note:
Info
If there is a stuck message again, just repeat the steps.
Alert
For the manually acknowledged message, the Provisioning Request Log will still show it as "In Progress" on the logs, this is for further investigation as needed.
As the review on the provisioning requests are done, there is a need to reprovision for skipped messages.

    • Related Articles

    • RabbitMQ on Atria v15

      RabbitMQ Overview RabbitMQ is an Open Source Message Broker, it is a lightweight and easy to deploy platform that enables Atria service components to securely communicate asynchronously with each other. RabbitMQ is a core part of Atria’s ...

    • How to Export Atria Provisioning Rules

      Overview This article is a guide on how to export the current provisioning rules in the environment. This will be used for the assessment as we go forward with Atria Migration or In-place Upgrade. Premise Run the script from a machine in the same ...

    • Moving RabbitMQ_BASEPATH to a Directory Instead of the Default User Profile

      Overview By default when installing and configuring RabbitMQ, the BASEPATH for all config, logs, plugins, and instances are pointed to the %appdata% of the User that did the installation. Using this article, the RABBITMQ_BASEPATH will be moved to a ...

    • Microsoft Online Service Provisioning

      Objective This article provides a guide to configuring and provisioning the Microsoft Online Service within the Atria Platform. Before you Start After setting up the connection to Partner Center, update the service' customer plan- Service Deployments ...

    • Microsoft Online - How to configure your Customer plan

      Overview The Microsoft Online service within Atria has been designed from the Ground Up with Customization and flexibility in mind. This means, that it can likely cater with most configuration scenarios you may come across with Microsoft Online ...