novaBBS - comp.os.vms - Re: BL860i2 blade unexplained system disk slowdowns

BL860i2 blade unexplained system disk slowdowns

<1d42207e-0a10-4c5e-a027-2b2e197afe88n@googlegroups.com>

https://www.novabbs.com/computers/article-flat.php?id=23079&group=comp.os.vms#23079

X-Received: by 2002:ac8:5c96:0:b0:2f9:3077:5cda with SMTP id r22-20020ac85c96000000b002f930775cdamr22911236qta.625.1654605299104;
Tue, 07 Jun 2022 05:34:59 -0700 (PDT)
X-Received: by 2002:a05:622a:6082:b0:2f1:1f9c:251e with SMTP id
hf2-20020a05622a608200b002f11f9c251emr22473516qtb.230.1654605298922; Tue, 07
Jun 2022 05:34:58 -0700 (PDT)
Path: i2pn2.org!i2pn.org!usenet.blueworldhosting.com!feed1.usenet.blueworldhosting.com!peer03.iad!feed-me.highwinds-media.com!news.highwinds-media.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.os.vms
Date: Tue, 7 Jun 2022 05:34:58 -0700 (PDT)
Injection-Info: google-groups.googlegroups.com; posting-host=56.0.143.28; posting-account=K_VWOAoAAAB-wPQ2QHcqIArSTrLiZ_w7
NNTP-Posting-Host: 56.0.143.28
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <1d42207e-0a10-4c5e-a027-2b2e197afe88n@googlegroups.com>
Subject: BL860i2 blade unexplained system disk slowdowns
From: kennethn...@gmail.com (Kenneth Randell)
Injection-Date: Tue, 07 Jun 2022 12:34:59 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Received-Bytes: 3301

by: Kenneth Randell - Tue, 7 Jun 2022 12:34 UTC

I have an issue with a system where the I/O to the system drive is after some period of time slowing down to the point of making the system unusable.

The hardware configuration is BL860i2 blade, VMS 8.4, 16GB memory, dual-processor quad-core CPUs, 2x1.2GB disks in Raid 1 configuration, p410i controller on the motherboard is running 6.64 firmware, no cache module

This system went through a power outage after a normal shutdown and power off. After it came back up, these system disk (the raid-1 pair in the blade) slow-downs began after the system has been up anywhere from 6-14 hours. XFC disk response times just start to increase and never decrease. The system starts off at around 500 Useconds, then at some point (for unknown reasons) will start increasing. Some time stamps below (this is 11 hours after the last shutdown and e-fuse).

7-JUN-2022 00:28:08.54 Ave Disk I/O Resp Time incl cache hits (microseconds) 385
7-JUN-2022 00:29:12.30 Ave Disk I/O Resp Time incl cache hits (microseconds) 471
7-JUN-2022 00:30:24.98 Ave Disk I/O Resp Time incl cache hits (microseconds) 559
7-JUN-2022 00:31:29.29 Ave Disk I/O Resp Time incl cache hits (microseconds) 669
7-JUN-2022 00:32:33.49 Ave Disk I/O Resp Time incl cache hits (microseconds) 811
7-JUN-2022 00:33:33.57 Ave Disk I/O Resp Time incl cache hits (microseconds) 912
7-JUN-2022 00:34:37.84 Ave Disk I/O Resp Time incl cache hits (microseconds) 1013

This system has been 'normal' for the last 6 years; I have not any idea why this has started now. The disk isn't fragmented to any degree; all 'working' storage is only other drives. There are no hardware faults - MSA$UTIL doesn't show anything out of the ordinary, etc. I have forced a system crash in the hopes that SDA would reveal something but that hasn't turned up anything. I have to assume something is going on in the hardware as I am at a loss to explain this from the O/S side.

Other blades with identical hardware/software/firmware and similar workloads have never shown this problem after a similar shutdown/power-outage/reboot scenario.

Any ideas?

Re: BL860i2 blade unexplained system disk slowdowns

<1498d691-f153-42cf-9212-e2ec1e9507e1n@googlegroups.com>

copy mid

https://www.novabbs.com/computers/article-flat.php?id=23080&group=comp.os.vms#23080

copy link Newsgroups: comp.os.vms

X-Received: by 2002:a05:622a:1009:b0:304:c115:907e with SMTP id d9-20020a05622a100900b00304c115907emr22569099qte.550.1654606205961;
Tue, 07 Jun 2022 05:50:05 -0700 (PDT)
X-Received: by 2002:ac8:578d:0:b0:2f3:e61d:df9e with SMTP id
v13-20020ac8578d000000b002f3e61ddf9emr22566523qta.181.1654606205814; Tue, 07
Jun 2022 05:50:05 -0700 (PDT)
Path: i2pn2.org!i2pn.org!usenet.blueworldhosting.com!feed1.usenet.blueworldhosting.com!peer03.iad!feed-me.highwinds-media.com!news.highwinds-media.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.os.vms
Date: Tue, 7 Jun 2022 05:50:05 -0700 (PDT)
In-Reply-To: <1d42207e-0a10-4c5e-a027-2b2e197afe88n@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=96.230.211.194; posting-account=Ysq9BAoAAACGX1EcMMPkdNg4YcTg0TxG
NNTP-Posting-Host: 96.230.211.194
References: <1d42207e-0a10-4c5e-a027-2b2e197afe88n@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <1498d691-f153-42cf-9212-e2ec1e9507e1n@googlegroups.com>
Subject: Re: BL860i2 blade unexplained system disk slowdowns
From: dansabrs...@yahoo.com (abrsvc)
Injection-Date: Tue, 07 Jun 2022 12:50:05 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Received-Bytes: 3654

by: abrsvc - Tue, 7 Jun 2022 12:50 UTC

On Tuesday, June 7, 2022 at 8:35:00 AM UTC-4, Kenneth Randell wrote:
> I have an issue with a system where the I/O to the system drive is after some period of time slowing down to the point of making the system unusable..
>
> The hardware configuration is BL860i2 blade, VMS 8.4, 16GB memory, dual-processor quad-core CPUs, 2x1.2GB disks in Raid 1 configuration, p410i controller on the motherboard is running 6.64 firmware, no cache module
>
> This system went through a power outage after a normal shutdown and power off. After it came back up, these system disk (the raid-1 pair in the blade) slow-downs began after the system has been up anywhere from 6-14 hours. XFC disk response times just start to increase and never decrease. The system starts off at around 500 Useconds, then at some point (for unknown reasons) will start increasing. Some time stamps below (this is 11 hours after the last shutdown and e-fuse).
>
>
> 7-JUN-2022 00:28:08.54 Ave Disk I/O Resp Time incl cache hits (microseconds) 385
> 7-JUN-2022 00:29:12.30 Ave Disk I/O Resp Time incl cache hits (microseconds) 471
> 7-JUN-2022 00:30:24.98 Ave Disk I/O Resp Time incl cache hits (microseconds) 559
> 7-JUN-2022 00:31:29.29 Ave Disk I/O Resp Time incl cache hits (microseconds) 669
> 7-JUN-2022 00:32:33.49 Ave Disk I/O Resp Time incl cache hits (microseconds) 811
> 7-JUN-2022 00:33:33.57 Ave Disk I/O Resp Time incl cache hits (microseconds) 912
> 7-JUN-2022 00:34:37.84 Ave Disk I/O Resp Time incl cache hits (microseconds) 1013
>
> This system has been 'normal' for the last 6 years; I have not any idea why this has started now. The disk isn't fragmented to any degree; all 'working' storage is only other drives. There are no hardware faults - MSA$UTIL doesn't show anything out of the ordinary, etc. I have forced a system crash in the hopes that SDA would reveal something but that hasn't turned up anything. I have to assume something is going on in the hardware as I am at a loss to explain this from the O/S side.
>
> Other blades with identical hardware/software/firmware and similar workloads have never shown this problem after a similar shutdown/power-outage/reboot scenario.
>
> Any ideas?
Check the cache battery status. If this has failed, the cache will be disabled completely and performance will suffer.

Dan

Re: BL860i2 blade unexplained system disk slowdowns

<jg90a2Fmf8vU1@mid.individual.net>

copy mid

https://www.novabbs.com/computers/article-flat.php?id=23081&group=comp.os.vms#23081

copy link Newsgroups: comp.os.vms

Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: use...@andyburns.uk (Andy Burns)
Newsgroups: comp.os.vms
Subject: Re: BL860i2 blade unexplained system disk slowdowns
Date: Tue, 7 Jun 2022 13:57:36 +0100
Lines: 8
Message-ID: <jg90a2Fmf8vU1@mid.individual.net>
References: <1d42207e-0a10-4c5e-a027-2b2e197afe88n@googlegroups.com>
<1498d691-f153-42cf-9212-e2ec1e9507e1n@googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Trace: individual.net 6jF3ivqWYinIAE8Xta4yYQyJIbNIoDEnoRnSyYdjop3nsC2ZxR
Cancel-Lock: sha1:5WhvmJUBwCtauTY9Sa7iwPcgK1o=
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.0
Content-Language: en-GB
In-Reply-To: <1498d691-f153-42cf-9212-e2ec1e9507e1n@googlegroups.com>

by: Andy Burns - Tue, 7 Jun 2022 12:57 UTC

abrsvc wrote:

> Check the cache battery status. If this has failed, the cache will be disabled completely and performance will suffer.

It would, except he said a battery backed cache module isn't fitted ...

Re: BL860i2 blade unexplained system disk slowdowns

<t7o217$a8o$1@dont-email.me>

copy mid

https://www.novabbs.com/computers/article-flat.php?id=23082&group=comp.os.vms#23082

copy link Newsgroups: comp.os.vms

Path: i2pn2.org!i2pn.org!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: club...@remove_me.eisner.decus.org-Earth.UFP (Simon Clubley)
Newsgroups: comp.os.vms
Subject: Re: BL860i2 blade unexplained system disk slowdowns
Date: Tue, 7 Jun 2022 17:31:19 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 34
Message-ID: <t7o217$a8o$1@dont-email.me>
References: <1d42207e-0a10-4c5e-a027-2b2e197afe88n@googlegroups.com>
Injection-Date: Tue, 7 Jun 2022 17:31:19 -0000 (UTC)
Injection-Info: reader02.eternal-september.org; posting-host="aa3c05e00c82c6a6fbedbde74c4fdff6";
logging-data="10520"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+n+nPggtC/U51GICd5ai7HeNWMGdVdeZk="
User-Agent: slrn/0.9.8.1 (VMS/Multinet)
Cancel-Lock: sha1:jqHUo13/tcXwAoC7IfFBZKub3yc=

by: Simon Clubley - Tue, 7 Jun 2022 17:31 UTC

On 2022-06-07, Kenneth Randell <kennethnospam.randell@gmail.com> wrote:
> I have an issue with a system where the I/O to the system drive is after some period of time slowing down to the point of making the system unusable.
>
> The hardware configuration is BL860i2 blade, VMS 8.4, 16GB memory, dual-processor quad-core CPUs, 2x1.2GB disks in Raid 1 configuration, p410i controller on the motherboard is running 6.64 firmware, no cache module
>
> This system went through a power outage after a normal shutdown and power off. After it came back up, these system disk (the raid-1 pair in the blade) slow-downs began after the system has been up anywhere from 6-14 hours. XFC disk response times just start to increase and never decrease. The system starts off at around 500 Useconds, then at some point (for unknown reasons) will start increasing. Some time stamps below (this is 11 hours after the last shutdown and e-fuse).
>
>
> 7-JUN-2022 00:28:08.54 Ave Disk I/O Resp Time incl cache hits (microseconds) 385
> 7-JUN-2022 00:29:12.30 Ave Disk I/O Resp Time incl cache hits (microseconds) 471
> 7-JUN-2022 00:30:24.98 Ave Disk I/O Resp Time incl cache hits (microseconds) 559
> 7-JUN-2022 00:31:29.29 Ave Disk I/O Resp Time incl cache hits (microseconds) 669
> 7-JUN-2022 00:32:33.49 Ave Disk I/O Resp Time incl cache hits (microseconds) 811
> 7-JUN-2022 00:33:33.57 Ave Disk I/O Resp Time incl cache hits (microseconds) 912
> 7-JUN-2022 00:34:37.84 Ave Disk I/O Resp Time incl cache hits (microseconds) 1013
>
> This system has been 'normal' for the last 6 years; I have not any idea why this has started now. The disk isn't fragmented to any degree; all 'working' storage is only other drives. There are no hardware faults - MSA$UTIL doesn't show anything out of the ordinary, etc. I have forced a system crash in the hopes that SDA would reveal something but that hasn't turned up anything. I have to assume something is going on in the hardware as I am at a loss to explain this from the O/S side.
>
> Other blades with identical hardware/software/firmware and similar workloads have never shown this problem after a similar shutdown/power-outage/reboot scenario.
>
> Any ideas?

Do the figures go back to normal _immediately_ after a reboot without
any power-off cycling or do you need to power off the machine and leave
it off for a while before they go back to normal ?

I am wondering if there's some subtle thermal-related issue due to
damage caused when the power outage occurred.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

Re: BL860i2 blade unexplained system disk slowdowns

<fe435b65-deaf-4f4b-a72f-436c8c7207c6n@googlegroups.com>

copy mid

https://www.novabbs.com/computers/article-flat.php?id=23087&group=comp.os.vms#23087

copy link Newsgroups: comp.os.vms

X-Received: by 2002:a05:622a:1983:b0:304:f190:1739 with SMTP id u3-20020a05622a198300b00304f1901739mr7501720qtc.382.1654629996827;
Tue, 07 Jun 2022 12:26:36 -0700 (PDT)
X-Received: by 2002:a05:620a:f0c:b0:6a6:31b2:b139 with SMTP id
v12-20020a05620a0f0c00b006a631b2b139mr21699818qkl.77.1654629996598; Tue, 07
Jun 2022 12:26:36 -0700 (PDT)
Path: i2pn2.org!i2pn.org!usenet.blueworldhosting.com!feed1.usenet.blueworldhosting.com!peer03.iad!feed-me.highwinds-media.com!news.highwinds-media.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.os.vms
Date: Tue, 7 Jun 2022 12:26:36 -0700 (PDT)
In-Reply-To: <t7o217$a8o$1@dont-email.me>
Injection-Info: google-groups.googlegroups.com; posting-host=56.0.143.28; posting-account=K_VWOAoAAAB-wPQ2QHcqIArSTrLiZ_w7
NNTP-Posting-Host: 56.0.143.28
References: <1d42207e-0a10-4c5e-a027-2b2e197afe88n@googlegroups.com> <t7o217$a8o$1@dont-email.me>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <fe435b65-deaf-4f4b-a72f-436c8c7207c6n@googlegroups.com>
Subject: Re: BL860i2 blade unexplained system disk slowdowns
From: kennethn...@gmail.com (Kenneth Randell)
Injection-Date: Tue, 07 Jun 2022 19:26:36 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Received-Bytes: 2841

by: Kenneth Randell - Tue, 7 Jun 2022 19:26 UTC

> Do the figures go back to normal _immediately_ after a reboot without
> any power-off cycling or do you need to power off the machine and leave
> it off for a while before they go back to normal ?
>
> I am wondering if there's some subtle thermal-related issue due to
> damage caused when the power outage occurred.
>
> Simon.
>
> --
> Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
> Walking destinations on a map are further away than they appear.

A reboot without power-cycle immediately clears the problem; no need to turn it off and leave it off for any length of time. According to the O/A in this C7000, there are no thermal issues; the ambient temperature of the blade sits at around 20 degrees C as shown below:

Locale Status Temp Caution Critical
----------------------------------- ------ -------- ------- --------
Ambient Zone (Ambient Temp ) OK 20C/ 68F 39C 42C
System Chassis (Top Exit Temp ) OK 35C/ 95F 72C 76C
System Chassis (Mid Exit Temp ) OK 44C/111F 75C 76C
System Chassis (Low Exit Temp ) OK 40C/104F 72C 76C
CPU Zone (Processor 0 Temp) OK 36C/ 96F 88C 91C
CPU Zone (Processor 1 Temp) OK 36C/ 96F 88C 91C
Memory Zone (DIMM1C-0 Temp ) N/A N/A
Memory Zone (DIMM2E-0 Temp ) N/A N/A
Memory Zone (DIMM3A-0 Temp ) OK 40C/104F 84C 85C
Memory Zone (DIMM4A-0 Temp ) OK 41C/105F 84C 85C

Re: BL860i2 blade unexplained system disk slowdowns

<t7q3t8$cer$3@dont-email.me>

copy mid

https://www.novabbs.com/computers/article-flat.php?id=23099&group=comp.os.vms#23099

copy link Newsgroups: comp.os.vms

Path: i2pn2.org!i2pn.org!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: club...@remove_me.eisner.decus.org-Earth.UFP (Simon Clubley)
Newsgroups: comp.os.vms
Subject: Re: BL860i2 blade unexplained system disk slowdowns
Date: Wed, 8 Jun 2022 12:15:36 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 22
Message-ID: <t7q3t8$cer$3@dont-email.me>
References: <1d42207e-0a10-4c5e-a027-2b2e197afe88n@googlegroups.com> <t7o217$a8o$1@dont-email.me> <fe435b65-deaf-4f4b-a72f-436c8c7207c6n@googlegroups.com>
Injection-Date: Wed, 8 Jun 2022 12:15:36 -0000 (UTC)
Injection-Info: reader02.eternal-september.org; posting-host="eede7606dd75c56058419036ded4829e";
logging-data="12763"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18TGplqpoI0PBeSRgz5V3wLn+WiZ8oxBZA="
User-Agent: slrn/0.9.8.1 (VMS/Multinet)
Cancel-Lock: sha1:7lmU2h27tuYM29M9mPqF19CynpI=

by: Simon Clubley - Wed, 8 Jun 2022 12:15 UTC

On 2022-06-07, Kenneth Randell <kennethnospam.randell@gmail.com> wrote:
>> Do the figures go back to normal _immediately_ after a reboot without
>> any power-off cycling or do you need to power off the machine and leave
>> it off for a while before they go back to normal ?
>>
>> I am wondering if there's some subtle thermal-related issue due to
>> damage caused when the power outage occurred.
>>
>
> A reboot without power-cycle immediately clears the problem; no need to turn it off and leave it off for any length of time. According to the O/A in this C7000, there are no thermal issues; the ambient temperature of the blade sits at around 20 degrees C as shown below:
>

I'll admit I'm out of ideas then. I think it's time for you to get
support involved if a reboot without a power-cycle immediately clears
the problem because that doesn't sound like a normal case of a failing
component gradually degrading during use unless its left to cool down.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

The road to hell is paved with NAND gates. -- J. Gooding

computers / comp.os.vms / Re: BL860i2 blade unexplained system disk slowdowns

computers / comp.os.vms / Re: BL860i2 blade unexplained system disk slowdowns

Subject	Author
BL860i2 blade unexplained system disk slowdowns	Kenneth Randell
Re: BL860i2 blade unexplained system disk slowdowns	abrsvc
Re: BL860i2 blade unexplained system disk slowdowns	Andy Burns
Re: BL860i2 blade unexplained system disk slowdowns	Simon Clubley
Re: BL860i2 blade unexplained system disk slowdowns	Kenneth Randell
Re: BL860i2 blade unexplained system disk slowdowns	Simon Clubley