Raghu

Just one item that you may want to check if this ever happens again,
I have seen it 'hang' a job before. Do a SHOWOUT job=hungjobnumber
and see if the job has any output devices opened HOT, not spooled.
If the device is not ready the job will hang until it becomes ready,
and then run slowly once it is read, being limited by the speed of the
device. You can abortio to the device, spool it back up and rerun
the job and all should be well.

Robert Schlosser
Global Business Systems
Premier Farnell plc
T 864 487-1362
F 864.487.1904
E rschlosser@premierfarnell.com
Your #1 source for RoHS compliant products & information.
www.newark.com



-----Original Message-----
From: HP-3000 Systems Discussion [mailto:HP3000-L@RAVEN.UTC.EDU]On
Behalf Of Rao, Raghu
Sent: Thursday, July 24, 2008 11:32 AM
To: HP3000-L@RAVEN.UTC.EDU
Subject: Re: [HP3000-L] Job not responding...


Special Thanks to everyone who responded.. While we have not yet
identified the exact source of the problem, re-running those same jobs
24 hours later worked. So, we concluded that it was not a software issue
but some hardware problem.

I should probably have provided more details, but you know what happens
when you only get 2 hours sleep in the past 48 hours.

The job was running a Cobol program that was accessing HEALTH image
database. The online backup was running simultaneously around 9:30 PM
when we thought this problem might have happened. But the online backup
later finished a little past midnight. All other production jobs were
running as usual without any issues. There were no locks on the
database. The job would not respond to any abort commands. SHOWPROC does
not show anything suspicious. GLANCE showed 0% CPU time and no other
commands under GLANCE for that PIN would show anything positive for us.
MPEX System volume set had 75% free space and PRODVOL had 50% free
space. I ran a test in our test accounts for those same claims and that
job completed fine in the estimated time, concluding that this was not a
software problem or claims related issue. While REBOOT worked, we don't
know if this is a permanent solution. May be we could have done a system
dump to capture what was going on, but we simply REBOOTed because we
were in a hurry to get production up and running for all our users, and
to be in a situation to think about what to do with our critical check
run process.

On the other hand HP support later called us back and wanted to login to
our systems to see what was happening behind the hung job, it was too
late because we had already rebooted by then. But we provided them login
details so that they can login to see the situation if it happens again.
Luckily for us, it did not happen..

Oh yeah.. !! It was raining that night, pouring actually.. There was
probably 4 - 5 lightning's per minute. My home phones and cable internet
was dead, so I had to drive down to work. I definitely saw a spike at my
residence last night for about quarter to half a second. My PC, Modem,
Wireless, TV, most of the big items got rebooted. The other small items
like my alarm clocks, microwave timers, etc did not feel the difference.

At this point, we have documented everything surrounding this situation
and are about to close this ticket assuming its an hardware problem and
not knowing if this would happen again.


> Raghu Rao
>
>
> _____________________________________________
> From: Rao, Raghu
> Sent: Wednesday, July 23, 2008 12:09 PM
> To: HP3000-L@RAVEN.UTC.EDU
> Subject: Job not responding...
>
> Hi all,
>
> We had a strange situation last night.. We are a Health Plan using
> Amisys software (not Amisys Advance). Our production box is a Hp3k
> N4000 series..
>
> One of the claims payment extract job which has been running fine so
> far from past 8 yrs, now all of sudden dies.. It was supposed to run
> for maximum 3 hours and kept on running for 8 hours. When I logged in
> at about 2:30 AM to view this job, there were no production jobs
> running at that time other than this hung job and our other regular
> system jobs. This hung job gets displayed on SHOWJOB command on the
> MPE prompt, however on GLANCE it shows 0% CPU time. This hung jobs did
> not respond to any commands. We aborted the job using regular ABORTJOB
> command and no response. The job still shows EXEC state. We tried
> ABORTPROC and it says that an abort is already pending on this job. We
> kept trying for 2 hours to get any response from this job and it was
> totally hung and non-responsive.
>
> Then we finally called HP at about 5:45 AM to figure out if they have
> any special ABORT JOB commands to kill this job. Their feedback was to
> SIMPLY REBOOT the machine, which was kind of surprising to us. But we
> had no other choice (as this job was one of our core jobs doing claims
> payment processing). We went ahead with the REBOOT which finally
> knocked off that job.
>
> But we are clueless as to what happened to this job... The job run
> STDLIST show blank after 8:12 PM. No other logs are showing anything
> positive. We investigated 4 production claims which this job could
> possibly be accessing at that particular moment when this job got
> hung, but further testing on those claims this morning revealed
> nothing fishy..
>
> Has anyone been through this deadlock before ? Any tips, pointers, etc
> would be really appreciated..
>
> Thanks and best regards.
>
> Raghu Rao
>



********************************************
This communication and any files or attachments transmitted with it may
contain information that is confidential, privileged and exempt from
disclosure under applicable law. It is intended solely for the use of the
individual or the entity to which it is addressed. If you are not the
intended recipient, you are hereby notified that any use, dissemination, or
copying of this communication is prohibited by federal law. If you have
received this communication in error, please destroy it and notify the
sender.
********************************************

* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *


https://www.mailcontrol.com/sr/wQw0zmjPoHdJTZGyOCrrhg==
u75fZZhccjq!x+2zbyCIMZsdq1iSCQwyNC8RpgWkw==
Click on the link above to report this email as spam to BlackSpider.


***************************Disclaimer************* **************
The contents of this e-mail and any file transmitted with it are confidential and intended solely for the individual or entity to whom they are addressed. The content may also contain legal, professional or other privileged information. If you received this e-mail in error, please destroy it immediately. You should not copy or use it for any purpose nor disclose its contents to any other person. The views stated herein do not necessarily represent the view of the Company. Please ensure you have adequate virus protection before you open or detach any documents from this transmission. The Company does not accept any liability for viruses.
Premier Farnell plc
150 Armley Road Leeds
LS12 2QQ
Telephone +44 (0) 870 129 8608
Fax +44 (0) 870 129 8610
Registered in England
Company Number 876412
Registered Office: Farnell House, Forge Lane, Leeds LS12 2NE ************************************************** **********

* To join/leave the list, search archives, change list settings, *
* etc., please visit http://raven.utc.edu/archives/hp3000-l.html *