Discussion:
Kernel 2.6.17 and RAID5 Grow Problem (critical section backup)
Justin Piszcz
2006-07-07 12:38:08 UTC
Permalink
p34:~# mdadm /dev/md3 -a /dev/hde1
mdadm: added /dev/hde1

p34:~# mdadm -D /dev/md3
/dev/md3:
Version : 00.90.03
Creation Time : Fri Jun 30 09:17:12 2006
Raid Level : raid5
Array Size : 1953543680 (1863.04 GiB 2000.43 GB)
Device Size : 390708736 (372.61 GiB 400.09 GB)
Raid Devices : 6
Total Devices : 7
Preferred Minor : 3
Persistence : Superblock is persistent

Update Time : Fri Jul 7 08:25:44 2006
State : clean
Active Devices : 6
Working Devices : 7
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 512K

UUID : e76e403c:7811eb65:73be2f3b:0c2fc2ce
Events : 0.232940

Number Major Minor RaidDevice State
0 22 1 0 active sync /dev/hdc1
1 56 1 1 active sync /dev/hdi1
2 3 1 2 active sync /dev/hda1
3 8 49 3 active sync /dev/sdd1
4 88 1 4 active sync /dev/hdm1
5 8 33 5 active sync /dev/sdc1

6 33 1 - spare /dev/hde1
p34:~# mdadm --grow /dev/md3 --raid-disks=7
mdadm: Need to backup 15360K of critical section..
mdadm: Cannot set device size/shape for /dev/md3: No space left on device
p34:~# mdadm --grow /dev/md3 --bitmap=internal --raid-disks=7
mdadm: can change at most one of size, raiddisks, bitmap, and layout
p34:~# umount /dev/md3
p34:~# mdadm --grow /dev/md3 --raid-disks=7
mdadm: Need to backup 15360K of critical section..
mdadm: Cannot set device size/shape for /dev/md3: No space left on device
p34:~#

The disk only has about 350GB of 1.8TB used, any idea why I get this
error?

I searched google but could not find anything on this issue when trying to
grow the array?


-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Justin Piszcz
2006-07-07 12:46:15 UTC
Permalink
Post by Justin Piszcz
p34:~# mdadm /dev/md3 -a /dev/hde1
mdadm: added /dev/hde1
p34:~# mdadm -D /dev/md3
Version : 00.90.03
Creation Time : Fri Jun 30 09:17:12 2006
Raid Level : raid5
Array Size : 1953543680 (1863.04 GiB 2000.43 GB)
Device Size : 390708736 (372.61 GiB 400.09 GB)
Raid Devices : 6
Total Devices : 7
Preferred Minor : 3
Persistence : Superblock is persistent
Update Time : Fri Jul 7 08:25:44 2006
State : clean
Active Devices : 6
Working Devices : 7
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 512K
UUID : e76e403c:7811eb65:73be2f3b:0c2fc2ce
Events : 0.232940
Number Major Minor RaidDevice State
0 22 1 0 active sync /dev/hdc1
1 56 1 1 active sync /dev/hdi1
2 3 1 2 active sync /dev/hda1
3 8 49 3 active sync /dev/sdd1
4 88 1 4 active sync /dev/hdm1
5 8 33 5 active sync /dev/sdc1
6 33 1 - spare /dev/hde1
p34:~# mdadm --grow /dev/md3 --raid-disks=7
mdadm: Need to backup 15360K of critical section..
mdadm: Cannot set device size/shape for /dev/md3: No space left on device
p34:~# mdadm --grow /dev/md3 --bitmap=internal --raid-disks=7
mdadm: can change at most one of size, raiddisks, bitmap, and layout
p34:~# umount /dev/md3
p34:~# mdadm --grow /dev/md3 --raid-disks=7
mdadm: Need to backup 15360K of critical section..
mdadm: Cannot set device size/shape for /dev/md3: No space left on device
p34:~#
The disk only has about 350GB of 1.8TB used, any idea why I get this error?
I searched google but could not find anything on this issue when trying to
grow the array?
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Is it because I use a 512kb chunksize?

Jul 7 08:44:59 p34 kernel: [4295845.933000] raid5: reshape: not enough
stripes. Needed 512
Jul 7 08:44:59 p34 kernel: [4295845.962000] md: couldn't update array
info. -28

So the RAID5 reshape only works if you use a 128kb or smaller chunk size?

Justin.
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Justin Piszcz
2006-07-07 12:49:43 UTC
Permalink
Post by Justin Piszcz
Post by Justin Piszcz
p34:~# mdadm /dev/md3 -a /dev/hde1
mdadm: added /dev/hde1
p34:~# mdadm -D /dev/md3
Version : 00.90.03
Creation Time : Fri Jun 30 09:17:12 2006
Raid Level : raid5
Array Size : 1953543680 (1863.04 GiB 2000.43 GB)
Device Size : 390708736 (372.61 GiB 400.09 GB)
Raid Devices : 6
Total Devices : 7
Preferred Minor : 3
Persistence : Superblock is persistent
Update Time : Fri Jul 7 08:25:44 2006
State : clean
Active Devices : 6
Working Devices : 7
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 512K
UUID : e76e403c:7811eb65:73be2f3b:0c2fc2ce
Events : 0.232940
Number Major Minor RaidDevice State
0 22 1 0 active sync /dev/hdc1
1 56 1 1 active sync /dev/hdi1
2 3 1 2 active sync /dev/hda1
3 8 49 3 active sync /dev/sdd1
4 88 1 4 active sync /dev/hdm1
5 8 33 5 active sync /dev/sdc1
6 33 1 - spare /dev/hde1
p34:~# mdadm --grow /dev/md3 --raid-disks=7
mdadm: Need to backup 15360K of critical section..
mdadm: Cannot set device size/shape for /dev/md3: No space left on device
p34:~# mdadm --grow /dev/md3 --bitmap=internal --raid-disks=7
mdadm: can change at most one of size, raiddisks, bitmap, and layout
p34:~# umount /dev/md3
p34:~# mdadm --grow /dev/md3 --raid-disks=7
mdadm: Need to backup 15360K of critical section..
mdadm: Cannot set device size/shape for /dev/md3: No space left on device
p34:~#
The disk only has about 350GB of 1.8TB used, any idea why I get this error?
I searched google but could not find anything on this issue when trying to
grow the array?
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Is it because I use a 512kb chunksize?
Jul 7 08:44:59 p34 kernel: [4295845.933000] raid5: reshape: not enough
stripes. Needed 512
Jul 7 08:44:59 p34 kernel: [4295845.962000] md: couldn't update array info.
-28
So the RAID5 reshape only works if you use a 128kb or smaller chunk size?
Justin.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
/* Can only proceed if there are plenty of stripe_heads.
@@ -2599,30 +2593,48 @@ static int raid5_reshape(mddev_t *mddev,
* If the chunk size is greater, user-space should request more
* stripe_heads first.
*/
- if ((mddev->chunk_size / STRIPE_SIZE) * 4 > conf->max_nr_stripes) {
+ if ((mddev->chunk_size / STRIPE_SIZE) * 4 > conf->max_nr_stripes ||
+ (mddev->new_chunk / STRIPE_SIZE) * 4 > conf->max_nr_stripes) {
printk(KERN_WARNING "raid5: reshape: not enough stripes. Needed %lu\n",
(mddev->chunk_size / STRIPE_SIZE)*4);
return -ENOSPC;
}

I don't see anything that mentions one needs to use a certain chunk size?

Any idea what the problem is here?

Justin.
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Justin Piszcz
2006-07-07 14:37:46 UTC
Permalink
Post by Justin Piszcz
Post by Justin Piszcz
Post by Justin Piszcz
p34:~# mdadm /dev/md3 -a /dev/hde1
mdadm: added /dev/hde1
p34:~# mdadm -D /dev/md3
Version : 00.90.03
Creation Time : Fri Jun 30 09:17:12 2006
Raid Level : raid5
Array Size : 1953543680 (1863.04 GiB 2000.43 GB)
Device Size : 390708736 (372.61 GiB 400.09 GB)
Raid Devices : 6
Total Devices : 7
Preferred Minor : 3
Persistence : Superblock is persistent
Update Time : Fri Jul 7 08:25:44 2006
State : clean
Active Devices : 6
Working Devices : 7
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 512K
UUID : e76e403c:7811eb65:73be2f3b:0c2fc2ce
Events : 0.232940
Number Major Minor RaidDevice State
0 22 1 0 active sync /dev/hdc1
1 56 1 1 active sync /dev/hdi1
2 3 1 2 active sync /dev/hda1
3 8 49 3 active sync /dev/sdd1
4 88 1 4 active sync /dev/hdm1
5 8 33 5 active sync /dev/sdc1
6 33 1 - spare /dev/hde1
p34:~# mdadm --grow /dev/md3 --raid-disks=7
mdadm: Need to backup 15360K of critical section..
mdadm: Cannot set device size/shape for /dev/md3: No space left on device
p34:~# mdadm --grow /dev/md3 --bitmap=internal --raid-disks=7
mdadm: can change at most one of size, raiddisks, bitmap, and layout
p34:~# umount /dev/md3
p34:~# mdadm --grow /dev/md3 --raid-disks=7
mdadm: Need to backup 15360K of critical section..
mdadm: Cannot set device size/shape for /dev/md3: No space left on device
p34:~#
The disk only has about 350GB of 1.8TB used, any idea why I get this error?
I searched google but could not find anything on this issue when trying to
grow the array?
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Is it because I use a 512kb chunksize?
Jul 7 08:44:59 p34 kernel: [4295845.933000] raid5: reshape: not enough
stripes. Needed 512
Jul 7 08:44:59 p34 kernel: [4295845.962000] md: couldn't update array
info. -28
So the RAID5 reshape only works if you use a 128kb or smaller chunk size?
Justin.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
/* Can only proceed if there are plenty of stripe_heads.
@@ -2599,30 +2593,48 @@ static int raid5_reshape(mddev_t *mddev,
* If the chunk size is greater, user-space should request more
* stripe_heads first.
*/
- if ((mddev->chunk_size / STRIPE_SIZE) * 4 > conf->max_nr_stripes) {
+ if ((mddev->chunk_size / STRIPE_SIZE) * 4 > conf->max_nr_stripes ||
+ (mddev->new_chunk / STRIPE_SIZE) * 4 > conf->max_nr_stripes) {
printk(KERN_WARNING "raid5: reshape: not enough stripes. Needed %lu\n",
(mddev->chunk_size / STRIPE_SIZE)*4);
return -ENOSPC;
}
I don't see anything that mentions one needs to use a certain chunk size?
Any idea what the problem is here?
Justin.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Neil,

Any comments?

Justin.
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Justin Piszcz
2006-07-07 19:04:37 UTC
Permalink
Post by Justin Piszcz
Post by Justin Piszcz
Post by Justin Piszcz
Post by Justin Piszcz
p34:~# mdadm /dev/md3 -a /dev/hde1
mdadm: added /dev/hde1
p34:~# mdadm -D /dev/md3
Version : 00.90.03
Creation Time : Fri Jun 30 09:17:12 2006
Raid Level : raid5
Array Size : 1953543680 (1863.04 GiB 2000.43 GB)
Device Size : 390708736 (372.61 GiB 400.09 GB)
Raid Devices : 6
Total Devices : 7
Preferred Minor : 3
Persistence : Superblock is persistent
Update Time : Fri Jul 7 08:25:44 2006
State : clean
Active Devices : 6
Working Devices : 7
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 512K
UUID : e76e403c:7811eb65:73be2f3b:0c2fc2ce
Events : 0.232940
Number Major Minor RaidDevice State
0 22 1 0 active sync /dev/hdc1
1 56 1 1 active sync /dev/hdi1
2 3 1 2 active sync /dev/hda1
3 8 49 3 active sync /dev/sdd1
4 88 1 4 active sync /dev/hdm1
5 8 33 5 active sync /dev/sdc1
6 33 1 - spare /dev/hde1
p34:~# mdadm --grow /dev/md3 --raid-disks=7
mdadm: Need to backup 15360K of critical section..
mdadm: Cannot set device size/shape for /dev/md3: No space left on device
p34:~# mdadm --grow /dev/md3 --bitmap=internal --raid-disks=7
mdadm: can change at most one of size, raiddisks, bitmap, and layout
p34:~# umount /dev/md3
p34:~# mdadm --grow /dev/md3 --raid-disks=7
mdadm: Need to backup 15360K of critical section..
mdadm: Cannot set device size/shape for /dev/md3: No space left on device
p34:~#
The disk only has about 350GB of 1.8TB used, any idea why I get this error?
I searched google but could not find anything on this issue when trying
to grow the array?
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Is it because I use a 512kb chunksize?
Jul 7 08:44:59 p34 kernel: [4295845.933000] raid5: reshape: not enough
stripes. Needed 512
Jul 7 08:44:59 p34 kernel: [4295845.962000] md: couldn't update array
info. -28
So the RAID5 reshape only works if you use a 128kb or smaller chunk size?
Justin.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
/* Can only proceed if there are plenty of stripe_heads.
@@ -2599,30 +2593,48 @@ static int raid5_reshape(mddev_t *mddev,
* If the chunk size is greater, user-space should request more
* stripe_heads first.
*/
- if ((mddev->chunk_size / STRIPE_SIZE) * 4 > conf->max_nr_stripes) {
+ if ((mddev->chunk_size / STRIPE_SIZE) * 4 > conf->max_nr_stripes ||
+ (mddev->new_chunk / STRIPE_SIZE) * 4 > conf->max_nr_stripes) {
printk(KERN_WARNING "raid5: reshape: not enough stripes. Needed %lu\n",
(mddev->chunk_size / STRIPE_SIZE)*4);
return -ENOSPC;
}
I don't see anything that mentions one needs to use a certain chunk size?
Any idea what the problem is here?
Justin.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Neil,
Any comments?
Justin.
The --grow option worked, sort of.

p34:~# mdadm /dev/md3 --grow --size=max
p34:~# umount /dev/md3
p34:~# mdadm -S /dev/md3
p34:~# mount /dev/md3
Segmentation fault
p34:~#

[4313355.425000] BUG: unable to handle kernel NULL pointer dereference at
virtual address 000000d4
[4313355.425000] printing eip:
[4313355.425000] c03c377b
[4313355.425000] *pde = 00000000
[4313355.425000] Oops: 0002 [#1]
[4313355.425000] PREEMPT SMP
[4313355.425000] CPU: 0
[4313355.425000] EIP: 0060:[<c03c377b>] Not tainted VLI
[4313355.425000] EFLAGS: 00010046 (2.6.17.3 #4)
[4313355.425000] EIP is at _spin_lock_irqsave+0x14/0x61
[4313355.425000] eax: 00000000 ebx: f7e6c000 ecx: c0333d12 edx:
00000202
[4313355.425000] esi: 000000d4 edi: f7fb9600 ebp: 000000d4 esp:
f7e6dc94
[4313355.425000] ds: 007b es: 007b ss: 0068
[4313355.425000] Process mount (pid: 22892, threadinfo=f7e6c000
task=c1a90580)
[4313355.425000] Stack: c19947e4 00000000 c0333d32 00000002 c012aaa2
f7e6dccc f7e6dc9c f7e6dc9c
[4313355.425000] f7e6dccc c0266b8d c19947e4 00000000 00000000
e11a61f8 f7e6dccc f7e6dccc
[4313355.425000] 00000005 f7fda014 f7fda000 f7fe8c00 c0259a79
e11a61c0 00000001 0000001f
[4313355.425000] Call Trace:
[4313355.425000] <c0333d32> raid5_unplug_device+0x20/0x65 <c012aaa2>
flush_workqueue+0x67/0x87
[4313355.425000] <c0266b8d> xfs_flush_buftarg+0x1ab/0x1c1 <c0259a79>
xfs_mount+0x322/0xbb5
[4313355.425000] <c013e3f2> truncate_inode_pages+0x2f/0x33 <c015c3e5>
set_blocksize+0x83/0x91
[4313355.425000] <c026d804> xfs_fs_fill_super+0x94/0x232 <c02825b3>
snprintf+0x2b/0x2f
[4313355.425000] <c0189c8e> disk_name+0xa9/0xb3 <c015c412>
sb_set_blocksize+0x1f/0x46
[4313355.425000] <c015b5ff> get_sb_bdev+0x100/0x13f <c016f104>
alloc_vfsmnt+0xab/0xd4
[4313355.425000] <c026ce9e> xfs_fs_get_sb+0x2f/0x33 <c026d770>
xfs_fs_fill_super+0x0/0x232
[4313355.425000] <c015abc4> do_kern_mount+0x49/0xc0 <c016ffc5>
do_mount+0x2c7/0x6e6
[4313355.425000] <c014372e> __handle_mm_fault+0x1fb/0x8a2 <c01439c2>
__handle_mm_fault+0x48f/0x8a2
[4313355.425000] <c013b6f5> __alloc_pages+0x53/0x2f1 <c013bbd2>
__get_free_pages+0x2d/0x4b
[4313355.425000] <c016ede0> copy_mount_options+0x2c/0x12e <c0170481>
sys_mount+0x9d/0xde
[4313355.425000] <c0102df3> syscall_call+0x7/0xb
[4313355.425000] Code: 85 c0 74 c3 f3 90 0f b6 06 84 c0 7e f0 eb b8 90 e8
60 f2 ff ff eb d1 56 53 89 c6 bb 00 e0 ff ff 21 e3 83 43 14 01 9c 5a fa 31
c0 <86> 06 84 c0 7e 0c c7 46 04 00 00 00 00 89 d0 5b 5e c3 52 9d 83
[4313355.425000] EIP: [<c03c377b>] _spin_lock_irqsave+0x14/0x61 SS:ESP
0068:f7e6dc94
[4313355.425000] <6>note: mount[22892] exited with preempt_count 1


-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Justin Piszcz
2006-07-07 19:42:09 UTC
Permalink
Post by Justin Piszcz
Post by Justin Piszcz
Post by Justin Piszcz
Post by Justin Piszcz
Post by Justin Piszcz
p34:~# mdadm /dev/md3 -a /dev/hde1
mdadm: added /dev/hde1
p34:~# mdadm -D /dev/md3
Version : 00.90.03
Creation Time : Fri Jun 30 09:17:12 2006
Raid Level : raid5
Array Size : 1953543680 (1863.04 GiB 2000.43 GB)
Device Size : 390708736 (372.61 GiB 400.09 GB)
Raid Devices : 6
Total Devices : 7
Preferred Minor : 3
Persistence : Superblock is persistent
Update Time : Fri Jul 7 08:25:44 2006
State : clean
Active Devices : 6
Working Devices : 7
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 512K
UUID : e76e403c:7811eb65:73be2f3b:0c2fc2ce
Events : 0.232940
Number Major Minor RaidDevice State
0 22 1 0 active sync /dev/hdc1
1 56 1 1 active sync /dev/hdi1
2 3 1 2 active sync /dev/hda1
3 8 49 3 active sync /dev/sdd1
4 88 1 4 active sync /dev/hdm1
5 8 33 5 active sync /dev/sdc1
6 33 1 - spare /dev/hde1
p34:~# mdadm --grow /dev/md3 --raid-disks=7
mdadm: Need to backup 15360K of critical section..
mdadm: Cannot set device size/shape for /dev/md3: No space left on device
p34:~# mdadm --grow /dev/md3 --bitmap=internal --raid-disks=7
mdadm: can change at most one of size, raiddisks, bitmap, and layout
p34:~# umount /dev/md3
p34:~# mdadm --grow /dev/md3 --raid-disks=7
mdadm: Need to backup 15360K of critical section..
mdadm: Cannot set device size/shape for /dev/md3: No space left on device
p34:~#
The disk only has about 350GB of 1.8TB used, any idea why I get this error?
I searched google but could not find anything on this issue when trying
to grow the array?
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Is it because I use a 512kb chunksize?
Jul 7 08:44:59 p34 kernel: [4295845.933000] raid5: reshape: not enough
stripes. Needed 512
Jul 7 08:44:59 p34 kernel: [4295845.962000] md: couldn't update array
info. -28
So the RAID5 reshape only works if you use a 128kb or smaller chunk size?
Justin.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
/* Can only proceed if there are plenty of stripe_heads.
@@ -2599,30 +2593,48 @@ static int raid5_reshape(mddev_t *mddev,
* If the chunk size is greater, user-space should request more
* stripe_heads first.
*/
- if ((mddev->chunk_size / STRIPE_SIZE) * 4 > conf->max_nr_stripes) {
+ if ((mddev->chunk_size / STRIPE_SIZE) * 4 > conf->max_nr_stripes ||
+ (mddev->new_chunk / STRIPE_SIZE) * 4 > conf->max_nr_stripes) {
printk(KERN_WARNING "raid5: reshape: not enough stripes. Needed %lu\n",
(mddev->chunk_size / STRIPE_SIZE)*4);
return -ENOSPC;
}
I don't see anything that mentions one needs to use a certain chunk size?
Any idea what the problem is here?
Justin.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Neil,
Any comments?
Justin.
The --grow option worked, sort of.
p34:~# mdadm /dev/md3 --grow --size=max
p34:~# umount /dev/md3
p34:~# mdadm -S /dev/md3
p34:~# mount /dev/md3
Segmentation fault
p34:~#
[4313355.425000] BUG: unable to handle kernel NULL pointer dereference at
virtual address 000000d4
[4313355.425000] c03c377b
[4313355.425000] *pde = 00000000
[4313355.425000] Oops: 0002 [#1]
[4313355.425000] PREEMPT SMP
[4313355.425000] CPU: 0
[4313355.425000] EIP: 0060:[<c03c377b>] Not tainted VLI
[4313355.425000] EFLAGS: 00010046 (2.6.17.3 #4)
[4313355.425000] EIP is at _spin_lock_irqsave+0x14/0x61
00000202
f7e6dc94
[4313355.425000] ds: 007b es: 007b ss: 0068
[4313355.425000] Process mount (pid: 22892, threadinfo=f7e6c000
task=c1a90580)
[4313355.425000] Stack: c19947e4 00000000 c0333d32 00000002 c012aaa2 f7e6dccc
f7e6dc9c f7e6dc9c
[4313355.425000] f7e6dccc c0266b8d c19947e4 00000000 00000000 e11a61f8
f7e6dccc f7e6dccc
[4313355.425000] 00000005 f7fda014 f7fda000 f7fe8c00 c0259a79 e11a61c0
00000001 0000001f
[4313355.425000] <c0333d32> raid5_unplug_device+0x20/0x65 <c012aaa2>
flush_workqueue+0x67/0x87
[4313355.425000] <c0266b8d> xfs_flush_buftarg+0x1ab/0x1c1 <c0259a79>
xfs_mount+0x322/0xbb5
[4313355.425000] <c013e3f2> truncate_inode_pages+0x2f/0x33 <c015c3e5>
set_blocksize+0x83/0x91
[4313355.425000] <c026d804> xfs_fs_fill_super+0x94/0x232 <c02825b3>
snprintf+0x2b/0x2f
[4313355.425000] <c0189c8e> disk_name+0xa9/0xb3 <c015c412>
sb_set_blocksize+0x1f/0x46
[4313355.425000] <c015b5ff> get_sb_bdev+0x100/0x13f <c016f104>
alloc_vfsmnt+0xab/0xd4
[4313355.425000] <c026ce9e> xfs_fs_get_sb+0x2f/0x33 <c026d770>
xfs_fs_fill_super+0x0/0x232
[4313355.425000] <c015abc4> do_kern_mount+0x49/0xc0 <c016ffc5>
do_mount+0x2c7/0x6e6
[4313355.425000] <c014372e> __handle_mm_fault+0x1fb/0x8a2 <c01439c2>
__handle_mm_fault+0x48f/0x8a2
[4313355.425000] <c013b6f5> __alloc_pages+0x53/0x2f1 <c013bbd2>
__get_free_pages+0x2d/0x4b
[4313355.425000] <c016ede0> copy_mount_options+0x2c/0x12e <c0170481>
sys_mount+0x9d/0xde
[4313355.425000] <c0102df3> syscall_call+0x7/0xb
[4313355.425000] Code: 85 c0 74 c3 f3 90 0f b6 06 84 c0 7e f0 eb b8 90 e8 60
f2 ff ff eb d1 56 53 89 c6 bb 00 e0 ff ff 21 e3 83 43 14 01 9c 5a fa 31 c0
<86> 06 84 c0 7e 0c c7 46 04 00 00 00 00 89 d0 5b 5e c3 52 9d 83
[4313355.425000] EIP: [<c03c377b>] _spin_lock_irqsave+0x14/0x61 SS:ESP
0068:f7e6dc94
[4313355.425000] <6>note: mount[22892] exited with preempt_count 1
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Deleting the old raid5 (had data backed up before I tried this), oh, and
incase you are wondering, I rebooted and everything was fine, but I think
the 512kb chunksize is my problem. So I am deleting this array and then I
am going to create the smallest raid5 possibly with 3 drives and then try
to grow again.



-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Neil Brown
2006-07-07 22:00:55 UTC
Permalink
Post by Justin Piszcz
Post by Justin Piszcz
Jul 7 08:44:59 p34 kernel: [4295845.933000] raid5: reshape: not enough
stripes. Needed 512
Jul 7 08:44:59 p34 kernel: [4295845.962000] md: couldn't update array
info. -28
So the RAID5 reshape only works if you use a 128kb or smaller chunk size?
Neil,
Any comments?
Yes. This is something I need to fix in the next mdadm.
You need to tell md/raid5 to increase the size of the stripe cache
before the grow can proceed. You can do this with

echo 600 > /sys/block/md3/md/stripe_cache_size

Then the --grow should work. The next mdadm will do this for you.

NeilBrown

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Justin Piszcz
2006-07-07 22:15:41 UTC
Permalink
Post by Neil Brown
Post by Justin Piszcz
Post by Justin Piszcz
Jul 7 08:44:59 p34 kernel: [4295845.933000] raid5: reshape: not enough
stripes. Needed 512
Jul 7 08:44:59 p34 kernel: [4295845.962000] md: couldn't update array
info. -28
So the RAID5 reshape only works if you use a 128kb or smaller chunk size?
Neil,
Any comments?
Yes. This is something I need to fix in the next mdadm.
You need to tell md/raid5 to increase the size of the stripe cache
before the grow can proceed. You can do this with
echo 600 > /sys/block/md3/md/stripe_cache_size
Then the --grow should work. The next mdadm will do this for you.
NeilBrown
Hey! You're awake :)

I am going to try it with just 64kb to prove to myself it works with that,
but then I will re-create the raid5 again like I had it before and attempt
it again, I did not see that documented anywhere!! Also, how do you use
the --backup-file option? Nobody seems to know!
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Neil Brown
2006-07-07 22:31:41 UTC
Permalink
Post by Justin Piszcz
Hey! You're awake :)
Yes, and thinking about breakfast (it's 8:30am here).
Post by Justin Piszcz
I am going to try it with just 64kb to prove to myself it works with that,
but then I will re-create the raid5 again like I had it before and attempt
it again, I did not see that documented anywhere!! Also, how do you use
the --backup-file option? Nobody seems to know!
man mdadm
--backup-file=
This is needed when --grow is used to increase the number of
raid-devices in a RAID5 if there are no spare devices avail-
able. See the section below on RAID_DEVICE CHANGES. The file
should be stored on a separate device, not on the raid array
being reshaped.


So e.g.
mdadm --grow /dev/md3 --raid-disk=7 --backup-file=/root/md3-backup

mdadm will copy the first few stripes to /root/md3-backup and start
the reshape. Once it gets past the critical section, mdadm will
remove the file.
If your system crashed during the critical section, then you wont be
able to assemble the array without providing the backup file:

e.g.
mdadm --assemble /dev/md3 --backup-file=/root/md3-backup /dev/sd[a-g]

NeilBrown
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Justin Piszcz
2006-07-07 22:35:14 UTC
Permalink
Post by Neil Brown
Post by Justin Piszcz
Hey! You're awake :)
Yes, and thinking about breakfast (it's 8:30am here).
Post by Justin Piszcz
I am going to try it with just 64kb to prove to myself it works with that,
but then I will re-create the raid5 again like I had it before and attempt
it again, I did not see that documented anywhere!! Also, how do you use
the --backup-file option? Nobody seems to know!
man mdadm
--backup-file=
This is needed when --grow is used to increase the number of
raid-devices in a RAID5 if there are no spare devices avail-
able. See the section below on RAID_DEVICE CHANGES. The file
should be stored on a separate device, not on the raid array
being reshaped.
So e.g.
mdadm --grow /dev/md3 --raid-disk=7 --backup-file=/root/md3-backup
mdadm will copy the first few stripes to /root/md3-backup and start
the reshape. Once it gets past the critical section, mdadm will
remove the file.
If your system crashed during the critical section, then you wont be
e.g.
mdadm --assemble /dev/md3 --backup-file=/root/md3-backup /dev/sd[a-g]
NeilBrown
Gotcha, thanks.

Quick question regarding reshaping, must one wait until the re-shape is
completed before he or she grows the file system?

With the re-shape still in progress, I tried to grow the xfs FS but it
stayed the same.

p34:~# df -h | grep /raid5
/dev/md3 746G 80M 746G 1% /raid5

p34:~# mdadm /dev/md3 --grow --raid-disks=4
mdadm: Need to backup 384K of critical section..
mdadm: ... critical section passed.
p34:~#

p34:~# cat /proc/mdstat
md3 : active raid5 hdc1[3] sdc1[2] hde1[1] hda1[0]
781417472 blocks super 0.91 level 5, 64k chunk, algorithm 2 [4/4]
[UUUU]
[>....................] reshape = 0.0% (85120/390708736)
finish=840.5min speed=7738K/sec
p34:~#

p34:~# mount /raid5
p34:~# xfs_growfs /raid5
meta-data=/dev/md3 isize=256 agcount=32, agsize=6104816
blks
= sectsz=4096 attr=0
data = bsize=4096 blocks=195354112, imaxpct=25
= sunit=16 swidth=48 blks, unwritten=1
naming =version 2 bsize=4096
log =internal bsize=4096 blocks=32768, version=2
= sectsz=4096 sunit=1 blks
realtime =none extsz=196608 blocks=0, rtextents=0
data blocks changed from 195354112 to 195354368
p34:~#

p34:~# umount /raid5
p34:~# mount /raid5
p34:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md3 746G 80M 746G 1% /raid5
p34:~#

I guess one has to wait until the reshape is complete before growing the
filesystem..?
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Neil Brown
2006-07-07 22:38:03 UTC
Permalink
Post by Justin Piszcz
I guess one has to wait until the reshape is complete before growing the
filesystem..?
Yes. The extra space isn't available until the reshape has completed
(if it was available earlier, the reshape wouldn't be necessary....)

NeilBrown
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Justin Piszcz
2006-07-07 22:39:32 UTC
Permalink
Post by Neil Brown
Post by Justin Piszcz
I guess one has to wait until the reshape is complete before growing the
filesystem..?
Yes. The extra space isn't available until the reshape has completed
(if it was available earlier, the reshape wouldn't be necessary....)
NeilBrown
Just wanted to confirm, thanks for all the help, I look forward to the new
revision of mdadm :) In the mean time, after I get another drive I will
try your work around, but so far it looks good, thanks.!

Justin.

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Justin Piszcz
2006-07-10 21:47:24 UTC
Permalink
Post by Neil Brown
Post by Justin Piszcz
Post by Justin Piszcz
Jul 7 08:44:59 p34 kernel: [4295845.933000] raid5: reshape: not enough
stripes. Needed 512
Jul 7 08:44:59 p34 kernel: [4295845.962000] md: couldn't update array
info. -28
So the RAID5 reshape only works if you use a 128kb or smaller chunk size?
Neil,
Any comments?
Yes. This is something I need to fix in the next mdadm.
You need to tell md/raid5 to increase the size of the stripe cache
before the grow can proceed. You can do this with
echo 600 > /sys/block/md3/md/stripe_cache_size
Then the --grow should work. The next mdadm will do this for you.
NeilBrown
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
md3 : active raid5 sdc1[7] sde1[6] sdd1[5] hdk1[2] hdi1[4] hde1[3] hdc1[1]
hda1[0]
2344252416 blocks super 0.91 level 5, 512k chunk, algorithm 2 [8/8]
[UUUUUUUU]
[>....................] reshape = 0.2% (1099280/390708736)
finish=1031.7min speed=6293K/sec

It is working, thanks!

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Jan Engelhardt
2006-07-10 22:27:14 UTC
Permalink
Post by Justin Piszcz
md3 : active raid5 sdc1[7] sde1[6] sdd1[5] hdk1[2] hdi1[4] hde1[3] hdc1[1]
hda1[0]
2344252416 blocks super 0.91 level 5, 512k chunk, algorithm 2 [8/8]
[UUUUUUUU]
[>....................] reshape = 0.2% (1099280/390708736)
finish=1031.7min speed=6293K/sec
It is working, thanks!
Hm, what's superblock 0.91? It is not mentioned in mdadm.8.


Jan Engelhardt
--
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Justin Piszcz
2006-07-10 22:30:35 UTC
Permalink
Post by Jan Engelhardt
Post by Justin Piszcz
md3 : active raid5 sdc1[7] sde1[6] sdd1[5] hdk1[2] hdi1[4] hde1[3] hdc1[1]
hda1[0]
2344252416 blocks super 0.91 level 5, 512k chunk, algorithm 2 [8/8]
[UUUUUUUU]
[>....................] reshape = 0.2% (1099280/390708736)
finish=1031.7min speed=6293K/sec
It is working, thanks!
Hm, what's superblock 0.91? It is not mentioned in mdadm.8.
Jan Engelhardt
--
Not sure, the block version perhaps?

I am using:

$ mdadm -V
mdadm - v2.5 - 26 May 2006

Debian Etch.

Justin.
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Jan Engelhardt
2006-07-11 07:52:00 UTC
Permalink
Post by Justin Piszcz
Post by Jan Engelhardt
Hm, what's superblock 0.91? It is not mentioned in mdadm.8.
Not sure, the block version perhaps?
Well yes of course, but what characteristics? The manual only lists

0, 0.90, default
Use the original 0.90 format superblock. This format
limits arrays to 28 componenet devices and limits compo■
nent devices of levels 1 and greater to 2 terabytes.

1, 1.0, 1.1, 1.2
Use the new version-1 format superblock. This has few
restrictions. The different subversion store the
superblock at different locations on the device, either
at the end (for 1.0), at the start (for 1.1) or 4K from
the start (for 1.2).

No 0.91 :(
(My mdadm is 2.2, but the problem remains in 2.5.2)


Jan Engelhardt
--
Petr Vyskocil
2006-07-11 09:05:59 UTC
Permalink
Post by Jan Engelhardt
Post by Justin Piszcz
Post by Jan Engelhardt
Hm, what's superblock 0.91? It is not mentioned in mdadm.8.
Not sure, the block version perhaps?
Well yes of course, but what characteristics? The manual only lists
0, 0.90, default
1, 1.0, 1.1, 1.2
No 0.91 :(
AFAICR superblock version gets raised by 0.01 for the duration of
reshape, so that non-reshape aware kernels do not try to assemble it
(and cause data corruption).

Petr

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Neil Brown
2006-07-18 00:16:27 UTC
Permalink
Post by Petr Vyskocil
Post by Jan Engelhardt
Post by Justin Piszcz
Post by Jan Engelhardt
Hm, what's superblock 0.91? It is not mentioned in mdadm.8.
Not sure, the block version perhaps?
Well yes of course, but what characteristics? The manual only lists
0, 0.90, default
1, 1.0, 1.1, 1.2
No 0.91 :(
AFAICR superblock version gets raised by 0.01 for the duration of
reshape, so that non-reshape aware kernels do not try to assemble it
(and cause data corruption).
Exactly. The following will be in the next mdadm - unless someone
wants to re-write it for me using shorter sentences :-)

NeilBrown



diff .prev/md.4 ./md.4
--- .prev/md.4 2006-06-20 10:01:17.000000000 +1000
+++ ./md.4 2006-07-18 10:14:47.000000000 +1000
@@ -74,6 +74,14 @@ UUID
a 128 bit Universally Unique Identifier that identifies the array that
this device is part of.

+When a version 0.90 array is being reshaped (e.g. adding extra devices
+to a RAID5), the version number is temporarily set to 0.91. This
+ensures that if the reshape process is stopped in the middle (e.g. by
+a system crash) and the machine boots into an older kernel that does
+not support reshaping, then the array will not be assembled (which
+would cause data corruption) but will be left untouched until a kernel
+that can complete the reshape processes is used.
+
.SS ARRAYS WITHOUT SUPERBLOCKS
While it is usually best to create arrays with superblocks so that
they can be assembled reliably, there are some circumstances where an
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Loading...