2012 run DAQ malfunction
2n correlation experiment
The experiment started on 08/02/2012 and ended on 08/24/2012.
08/08/2012 - switch to "common stop" mode. No crashes detected so far.
08/16/2012 - First crash observed on the first run on that day. Several minutes later after "beam down" failure another crash. 10:50 am another crash. 3 crashes in total.
08/17/2012 - the crash happend 3 times.
08/20/2012 - the crash happend 4 times.
08/22/2012 - the crash happend 2 times.
08/23/2012 - the crash happend 14 times. See the details below.
08/24/2012 - the crash happend 4 times.
ROC crash reports (08/23/2012)
from: Oleksiy Kosinov <kosiolek@isu.edu> to: Tony Forest <foretony@isu.edu> date: Thu, Aug 23, 2012 at 11:09 AM subject: crash_report mailed-by: isu.edu
Hello Dr. Forest,
Here is daq crash report:
INFO : transition Prestart succeeded.
INFO : LDS_ER go.....
INFO : eb1 go.....
INFO : roc1 go.....
INFO : transition Go succeeded.
WARN : roc1 has not reported status for 21 seconds
WARN : roc1 has not reported status for 31 seconds
WARN : roc1 has not reported status for 41 seconds
WARN : roc1 has not reported status for 51 seconds
WARN : roc1 has not reported status for 61 seconds
WARN : roc1 has not reported status for 71 seconds
WARN : roc1 has not reported status for 81 seconds
ERROR : End Run, roc1 no status for 141 seconds
ERROR : roc1 is in state disconnected should be active
WARN : roc1 has not reported status for 91 seconds
ERROR : roc1 is in state disconnected should be active
WARN : roc1 has not reported status for 101 seconds
ERROR : roc1 is in state disconnected should be active
WARN : roc1 has not reported status for 111 seconds
ERROR : roc1 is in state disconnected should be active
WARN : roc1 has not reported status for 121 seconds
ERROR : roc1 is in state disconnected should be active
WARN : roc1 has not reported status for 131 seconds
ERROR : roc1 is in state disconnected should be active
WARN : roc1 has not reported status for 141 seconds
ERROR : roc1 is in state disconnected should be active
WARN : roc1 has not reported status for 151 seconds
ERROR : roc1 is in state disconnected should be active
WARN : roc1 has not reported status for 161 seconds
ERROR : roc1 is in state disconnected should be active
WARN : roc1 has not reported status for 171 seconds
ERROR : roc1 is in state disconnected should be active
from: Tony Forest <foretony@isu.edu>
to: Oleksiy Kosinov <kosiolek@isu.edu>
date: Thu, Aug 23, 2012 at 11:22 AM
subject: Re: crash_report
That is a ROC crash not a DAQ host computer (daq1) crash.
You should only need to reboot the ROC in the VME crate.
from: Oleksiy Kosinov <kosiolek@isu.edu>
to: Tony Forest <foretony@isu.edu>
date: Thu, Aug 23, 2012 at 11:27 AM
subject: Re: crash_report
There is another crash. 2 min ago.
FO : roc1 go.....
INFO : transition Go succeeded.
WARN : roc1 has not reported status for 16 seconds
WARN : roc1 has not reported status for 26 seconds
WARN : roc1 has not reported status for 36 seconds
WARN : roc1 has not reported status for 46 seconds
WARN : roc1 has not reported status for 56 seconds
WARN : roc1 has not reported status for 66 seconds
WARN : roc1 has not reported status for 76 seconds
ERROR : End Run, roc1 no status for 136 seconds
ERROR : roc1 is in state disconnected should be active
WARN : roc1 has not reported status for 86 seconds
ERROR : roc1 is in state disconnected should be active
Thanks,
Oleksiy
from: Oleksiy Kosinov <kosiolek@isu.edu>
to: Tony Forest <foretony@isu.edu>
date: Thu, Aug 23, 2012 at 11:58 AM
subject: Re: crash_report
mailed-by: isu.edu
Another crash:
INFO : LDS_ER go.....
INFO : eb1 go.....
INFO : roc1 go.....
INFO : transition Go succeeded.
WARN : roc1 has not reported status for 13 seconds
WARN : roc1 has not reported status for 23 seconds
WARN : roc1 has not reported status for 33 seconds
WARN : roc1 has not reported status for 43 seconds
WARN : roc1 has not reported status for 48 seconds
INFO : roc1 end......
WARN : roc1 has not reported status for 13 seconds
WARN : roc1 has not reported status for 14 seconds
WARN : roc1 has not reported status for 15 seconds
WARN : roc1 has not reported status for 15 seconds
WARN : roc1 has not reported status for 16 seconds
WARN : roc1 has not reported status for 17 secon
ROC window message:
daLogMsg: INFO: prestarted
daLogMsg: INFO: activating
informEB: msgQSend done...
rolp->daproc = 5
daLogMsg: INFO: Entering User Go 2
rolp->daproc = 5
daLogMsg: INFO: Entering User Go
daLogMsg: INFO: active, events so far 0 token 0
->