Monitoring Tessitura Report Server state

Hi Jason,

We have similar issues where Report Server stops working and no one really knows until scheduled jobs are missed. I am not aware of a good way to monitor the actual state of the report server application itself; however, I am currently planning on creating a scheduled job (likely via SQL Server Agent) that essentially checks the scheduled queue table – which I believe is gooesoft_schedule_queue:

select COUNT(*)

  from gooesoft_schedule_queue a

where a.status = 's'

   and DATEADD(HH, 1, a.computed_date) < GETDATE();

If this query returns anything other than 0, then the report server is likely hung and my job would send an email out to administrators. I am padding this by an hour to take into account delays in starting jobs, as well as the fact that some jobs may not run exactly when scheduled due to the fact that multiple jobs are scheduled at around the same time. I’ll probably adjust how much time I pad this by based on how many false positives come out once I first implement. 

Thanks,

David

_______________________________________________________
David Frederick
Database Applications Analyst
Segerstrom Center for the Arts
600 Town Center Drive, Costa Mesa, CA 92626
T (714) 556-2122 x 4067 E DFrederick@SCFTA.org

We periodically run into issues where the Tessitura Report Server is in an "decided to stop process scheduled reports" state.  When you log into the server, you will see that the Report Server is still running (not frozen or anything like that), but it has stopped processing scheduled jobs due to any number of possible issues which trigger a pop-up window describing the actual issue. Off the top of my head, a couple of problems I've found that will cause this state are deadlocks; dropped connection to Tessitura services, which for some reason the Report Server will not attempt to reconnect to; and a report having an invalid parameter. But there are more.

We've been struggling with identifying when the Report Server application is in this state. Sometimes it is days before someone realizes their reports are not running and by then it is too late in some cases...

Does anyone know of a good way to monitor the state of the Report Server application?

Ultimately I'd like to set up more intelligent alerting so we know when reports are not processing. One idea is to simply have a report scheduled to run every morning and if IT does not receive this, then we know there was an issue. The problem with this is that we have to remember to look for this email every morning and if it is missing.  Also it is not a perfect solution as a time-sensative report may still be missed.  I also saw a post referencing the following app: http://jockersoft.com/english/appmonitor_index.php   I am unsure if this will help though because technically the Report Server is still running.

This message was sent automatically to you by www.tessituranetwork.com because you subscribed to the Tessitura Technical Forum. You may reply to this message to post to the Technical forum or visit the site to search, read and post to the forums. In the interest of keeping the forum posts from becoming cluttered, we encourage you to delete previous message text from your reply before sending. Thank you!

Hi Jason,

We have similar issues where Report Server stops working and no one really knows until scheduled jobs are missed. I am not aware of a good way to monitor the actual state of the report server application itself; however, I am currently planning on creating a scheduled job (likely via SQL Server Agent) that essentially checks the scheduled queue table – which I believe is gooesoft_schedule_queue:

select COUNT(*)

  from gooesoft_schedule_queue a

where a.status = 's'

   and DATEADD(HH, 1, a.computed_date) < GETDATE();

If this query returns anything other than 0, then the report server is likely hung and my job would send an email out to administrators. I am padding this by an hour to take into account delays in starting jobs, as well as the fact that some jobs may not run exactly when scheduled due to the fact that multiple jobs are scheduled at around the same time. I’ll probably adjust how much time I pad this by based on how many false positives come out once I first implement. 

Thanks,

David

_______________________________________________________
David Frederick
Database Applications Analyst
Segerstrom Center for the Arts
600 Town Center Drive, Costa Mesa, CA 92626
T (714) 556-2122 x 4067 E DFrederick@SCFTA.org

We periodically run into issues where the Tessitura Report Server is in an "decided to stop process scheduled reports" state.  When you log into the server, you will see that the Report Server is still running (not frozen or anything like that), but it has stopped processing scheduled jobs due to any number of possible issues which trigger a pop-up window describing the actual issue. Off the top of my head, a couple of problems I've found that will cause this state are deadlocks; dropped connection to Tessitura services, which for some reason the Report Server will not attempt to reconnect to; and a report having an invalid parameter. But there are more.

We've been struggling with identifying when the Report Server application is in this state. Sometimes it is days before someone realizes their reports are not running and by then it is too late in some cases...

Does anyone know of a good way to monitor the state of the Report Server application?

Ultimately I'd like to set up more intelligent alerting so we know when reports are not processing. One idea is to simply have a report scheduled to run every morning and if IT does not receive this, then we know there was an issue. The problem with this is that we have to remember to look for this email every morning and if it is missing.  Also it is not a perfect solution as a time-sensative report may still be missed.  I also saw a post referencing the following app: http://jockersoft.com/english/appmonitor_index.php   I am unsure if this will help though because technically the Report Server is still running.

This message was sent automatically to you by www.tessituranetwork.com because you subscribed to the Tessitura Technical Forum. You may reply to this message to post to the Technical forum or visit the site to search, read and post to the forums. In the interest of keeping the forum posts from becoming cluttered, we encourage you to delete previous message text from your reply before sending. Thank you!