How can I attach a disk to existing ZFS zpool mirror with Ubuntu and ZEVO?
質問
Is it possible to attach a third hdd to a already created mirror consisting of two hdds with ZEVO?
Ubuntu
I run an Ubuntu Virtual Machine with native ZFS support. When I attach the third hdd it works flawlessly.
OS X Lion
When I import the pool to Lion, ZEVO argues that one pool is missing even though it is there and gets resilvered at the moment.
On Ubuntu:
$ zpool status Toshiba_ZFS
pool: Toshiba_ZFS
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Sun Aug 19 15:03:18 2012
12.1G scanned out of 598G at 14.4M/s, 11h33m to go
929M resilvered, 2.02% done
config:
NAME STATE READ WRITE CKSUM
Toshiba_ZFS ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
sdd ONLINE 0 0 0
sdc ONLINE 0 0 0
sdb5 ONLINE 0 0 5 (resilvering)
On Mac OS X (Lion):
$ zpool status Toshiba_ZFS
pool: Toshiba_ZFS
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Sun Aug 19 15:03:18 2012
1,52Gi scanned out of 598Gi at 24,0Mi/s, 7h4m to go
242Mi resilvered, 0,25% done
config:
NAME STATE READ WRITE CKSUM
Toshiba_ZFS DEGRADED 0 0 0
mirror-0 DEGRADED 0 0 0
GPTE_1C3475D8-AB6F-3547-AE5D-571C2389DCC7 ONLINE 0 0 0 at disk3s1
GPTE_11059782-DA42-654B-8577-431C1B80814C ONLINE 0 0 0 at disk5s1
16494388674814556229 UNAVAIL 0 0 0 was /dev/sdb5
For some reason ZEVO stores the pools in /dev/dsk/ but since my third hdd isnt a pool yet and will be attached to existing zfs mirror pool it wont get recognised
$diskutil info disk2s5
Device Identifier: disk2s5
Device Node: /dev/disk2s5
Part of Whole: disk2
Device / Media Name: tank
Volume Name: Not applicable (no file system)
Mounted: Not applicable (no file system)
File System: None
Partition Type: FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF
OS Can Be Installed: No
Media Type: Generic
Protocol: USB
SMART Status: Not Supported
Total Size: 751.6 GB (751591690240 Bytes) (exactly 1467952520 512-Byte-Blocks)
Volume Free Space: Not applicable (no file system)
Device Block Size: 512 Bytes
Read-Only Media: No
Read-Only Volume: Not applicable (no file system)
Ejectable: Yes
Whole: No
Internal: No[/code]
$ zpool attach Toshiba_ZFS disk3s1 disk2s5
cannot open 'disk2s5': no such device in /dev/dsk
must be a full path or shorthand device name
How can I attach another disk with ZEVO?
EDIT: Here is some log information from the Console application:
20.08.12 09:52:01,000 kernel: ZFSLabelScheme:start: 0xffffff801160e700 created proxy disk for pool 'Toshiba_ZFS'
20.08.12 09:52:01,000 kernel: ZFSLabelScheme:start: 'Toshiba_ZFS' critical mass with 1 vdev(s) (skip import)
20.08.12 09:52:01,000 kernel: ZFSLabelScheme:probe: label 'Toshiba_ZFS', vdev 8248481474682216015
20.08.12 09:52:01,000 kernel: zfsx_vdm_open: couldn't find vdevMedia for 'sde5'
20.08.12 09:52:01,000 kernel: ldi_open_by_name: Toshiba_ZFS /dev/sde5 error 2, flag 3
20.08.12 09:52:01,000 kernel: zfsx_vdm_open: 'Toshiba_ZFS' disk3s1
20.08.12 09:52:01,000 kernel: zfsx_vdm_open: 'Toshiba_ZFS' disk5s1
20.08.12 09:52:02,000 kernel: zfsx_vdm_open: couldn't find vdevMedia for 'sde5'
20.08.12 09:52:02,000 kernel: ldi_open_by_name: Toshiba_ZFS /dev/sde5 error 2, flag 3
20.08.12 09:52:02,000 kernel: zfsx_mount: '/Volumes/Toshiba_ZFS'
20.08.12 09:52:07,235 com.tenscomplement.zfs.delegate: taking GPTE_11059782-DA42-654B- 8577-431C1B80814C online in pool 'Toshiba_ZFS'
20.08.12 09:52:17,239 com.tenscomplement.zfs.delegate: post mount fsgetpath: err 2, using "/.vol/234881038/2" instead
20.08.12 09:52:17,239 com.tenscomplement.zfs.delegate: post mount processing "/.vol/234881038/2"
And the partition table from parted:
$ sudo parted /dev/sdb
GNU Parted 2.3
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Model: External RAID (scsi)
Disk /dev/sdb: 1000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 20.5kB 210MB 210MB fat32 EFI System Partition boot
2 210MB 58.6GB 58.3GB Macintosh HD
3 58.7GB 59.3GB 650MB hfs+ Recovery HD
4 59.3GB 248GB 189GB hfs+ Backups
5 248GB 1000GB 752GB zfs Apple_HFS_Untitled_4
解決
Found a workaround. You will have to replicate the current zpool device's partition table to the device you want to attach and then randomize the unique GUIDs of the partitions (should be two). Then you add the device to the zpool mirror and format the unallocated disk space to HFS+ or whatever you like. This all happens inside Linux with gdisk and parted but it will be recognized properly by Mac OS X Zevos ZFS port.
$ zpool status Toshiba_ZFS
NAME STATE READ WRITE CKSUM
Toshiba_ZFS ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
GPTE_1C3475D8-AB6F-3547-AE5D-571C2389DCC7 ONLINE 0 0 0 at disk4s1
GPTE_11059782-DA42-654B-8577-431C1B80814C ONLINE 0 0 0 at disk3s1
GPTE_79E12ED8-31EE-384C-B115-2759039256C0 ONLINE 0 0 0 at disk2s1
$ diskutil list
/dev/disk2
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *1.0 TB disk2
1: ZFS 750.1 GB disk2s1
2: 6A945A3B-1DD2-11B2-99A6-080020736631 8.4 MB disk2s2
3: Apple_CoreStorage 60.5 GB disk2s3
4: Apple_Boot Recovery HD 650.0 MB disk2s4
5: EFI 209.7 MB disk2s5
6: Apple_HFS Backups 188.4 GB disk2s6
$ gdisk /dev/disk2
GPT fdisk (gdisk) version 0.8.2
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Command (? for help): p
Disk /dev/disk2: 1953525168 sectors, 931.5 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): B51307F7-D090-E64A-ADDF-800BFE2EA9AB
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1953525134
Partitions will be aligned on 32-sector boundaries
Total free space is 526309 sectors (257.0 MiB)
Number Start (sector) End (sector) Size Code Name
1 2048 1465131007 698.6 GiB BF01 zfs
2 1465131008 1465147391 8.0 MiB BF07
3 1465147392 1583354079 56.4 GiB AF05 Macintosh_HD
4 1583616224 1584885759 619.9 MiB AB00 Recovery HD
5 1584885760 1585295359 200.0 MiB EF00 EFI System Partition
6 1585295360 1953262983 175.5 GiB AF00
他のヒント
… For some reason zevo stores the pools in /dev/dsk/
In the new ZEVO knowledge base (2012-09-14), Disk device names provides the explanation.
A comparable explanation was published when Ten's Complement sold ZEVO.
How can I attach a disk to existing zfs zpool mirror with Zevo?
… Ubuntu … When I attach the third hdd it works flawlessly but when I import the pool to Lion …
In the knowledge base, Creating ZFS pools offers three examples that relate to mirrors.
From the ZEVO and OS X perspectives, it's almost certainly better to begin ZFS-related routines with ZEVO than with Ubuntu. Quoting from the QuickStart Guide for ZEVO Community Edition 1.1:
… ZEVO uses the standard ZFS on-disk format (v28) and is therefore binary compatible with ZFS on other platforms. However, direct interchange with other platforms is not supported in this version. ….
A comparable note was published when Ten's Complement sold ZEVO.
Starting points
http://zevo.getgreenbytes.com links to the knowledge base, support forum and more.