vmware:iscsi_target_for_shared_disks
Unterschiede
Hier werden die Unterschiede zwischen zwei Versionen angezeigt.
Beide Seiten der vorigen RevisionVorhergehende Überarbeitung | |||
vmware:iscsi_target_for_shared_disks [2018/06/02 18:16] – [ASM Platten hinzufügen] gpipperr | vmware:iscsi_target_for_shared_disks [2018/06/02 18:21] (aktuell) – [ASM Platten hinzufügen] gpipperr | ||
---|---|---|---|
Zeile 1: | Zeile 1: | ||
+ | ====== iSCSI Target für das Bereitstellen von Cluster Platten in VMWare | ||
+ | |||
+ | **01.2015 - Update 09.2016** | ||
+ | |||
+ | |||
+ | Siehe für Linux 6 die älteren Anmerkungen zu diesem Thema => [[vmware: | ||
+ | |||
+ | |||
+ | == Intention== | ||
+ | |||
+ | Folgende Problematik hat sich bei mir mit VMWare Workstation 10 ergeben: | ||
+ | |||
+ | Shared Disk möglich ABER => Gleichzeitiges Öffnen vom Controlfile im Oracle Rac Cluster führt zum Fehler. | ||
+ | |||
+ | Läuft nur einer der beiden Knoten, kann der jeweilige Knoten problemlos das Controlfile auf einer gemeinsame ASM Platte verwenden, laufen beide Knoten kann der Knoten 2 die Platte lesen, auf die Platte schreiben aber NICHT das Controlfile öffnen. | ||
+ | |||
+ | => Wir brauchen ein Shared Storage für eine echte VMWare Oracle Cluster Lösung | ||
+ | |||
+ | |||
+ | ===Terminologie=== | ||
+ | |||
+ | |||
+ | * iSCSI = Internet Small Computer System Interface | ||
+ | * LUN = Logical Unit Number - virtuelle Festplatte aus Sicht eines Betriebssystems | ||
+ | * iSCSI Target | ||
+ | * iSCSI Initiatoren | ||
+ | === Aufgabe== | ||
+ | Folgende Lun’s | ||
+ | |||
+ | {{ : | ||
+ | |||
+ | |||
+ | Die " | ||
+ | |||
+ | |||
+ | ---- | ||
+ | |||
+ | ====Das Betriebssystem installieren und vorbereiten==== | ||
+ | |||
+ | ==Grundinstallation== | ||
+ | |||
+ | Zum Einsatz kommt ein Oracle Linux 7.0 - minimale Grundinstallation ohne X, allerdings manuell die Platte konfiguriert, | ||
+ | |||
+ | |||
+ | Alternativ könnten auch eine NAS Appliance wie [[http:// | ||
+ | |||
+ | Allerdings benötigt der FreeNAS ~8GB RAM, was die mögliche Anzahl von gleichzeitigen VM’s auf einem Notebook | ||
+ | |||
+ | Per Yum Update dann am 22.03.2015 auf Oracle 7.1 upgedated, soweit keine Probleme aufgetaucht. | ||
+ | |||
+ | |||
+ | ==Weitere Schritte== | ||
+ | |||
+ | * Yum Repository prüfen, Konfiguration bei Bedarf anlegen (normalerweise bereits automatisch erfolgt( <code bash> | ||
+ | # cd / | ||
+ | # wget http:// | ||
+ | </ | ||
+ | * Update mit <code bash>yum update</ | ||
+ | * Firewalld unter linux7 deaktivieren :<code bash> | ||
+ | systemctl disable firewalld | ||
+ | systemctl stop firewalld | ||
+ | </ | ||
+ | * IP V6 ausschalten< | ||
+ | vi / | ||
+ | # disable ipv6 | ||
+ | net.ipv6.conf.all.disable_ipv6 = 1 | ||
+ | net.ipv6.conf.default.disable_ipv6 = 1 | ||
+ | net.ipv6.conf.lo.disable_ipv6 = 1 | ||
+ | |||
+ | sysctl -p | ||
+ | |||
+ | #Testen mit: | ||
+ | |||
+ | cat / | ||
+ | |||
+ | 1 | ||
+ | |||
+ | # 1=Ausgeschaltet! | ||
+ | </ | ||
+ | * SELinux deaktivieren:< | ||
+ | vi / | ||
+ | .. | ||
+ | SELINUX=disabled | ||
+ | .. | ||
+ | |||
+ | reboot | ||
+ | |||
+ | getenforce | ||
+ | |||
+ | Disabled | ||
+ | |||
+ | </ | ||
+ | * Netzwerk einrichten und darauf achten das in etc/hosts der Server in der Form " | ||
+ | * Zeitdienst einrichten , siehe auch => [[linux: | ||
+ | * Interface Name wieder auf eth0 konfigurieren, | ||
+ | * Falls das „tmp“ Verzeichnis als eigene Disk gemountet ist, darauf achten das " | ||
+ | |||
+ | |||
+ | |||
+ | <fc # | ||
+ | Ab der Linux 7 Versionen ist der ifconfig Befehlt mit pifconfig ersetzt worden! | ||
+ | |||
+ | |||
+ | ---- | ||
+ | |||
+ | |||
+ | ==== Die Lun‘s in einem eigenen Volumen anlegen ==== | ||
+ | |||
+ | Die per iSCSI dann zur Verfügung gestellten Platten liegen in einem eigenen Volumen. | ||
+ | |||
+ | Ablauf: | ||
+ | |||
+ | * Eine neue Platte mit 150GB an die Maschine " | ||
+ | * LVM „volume group“ anlegen< | ||
+ | vgcreate racstore01 /dev/sdb | ||
+ | </ | ||
+ | * LVM logical volume für jede benötigte RAC Lun anlegen< | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | |||
+ | #Devices dazu identifizieren: | ||
+ | cd / | ||
+ | ls -l | ||
+ | lrwxrwxrwx | ||
+ | lrwxrwxrwx | ||
+ | lrwxrwxrwx | ||
+ | lrwxrwxrwx | ||
+ | lrwxrwxrwx | ||
+ | lrwxrwxrwx | ||
+ | lrwxrwxrwx | ||
+ | lrwxrwxrwx | ||
+ | lrwxrwxrwx | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | ---- | ||
+ | |||
+ | |||
+ | ==== Breitstellen der Cluster Platten über iSCSI - Konfiguration der Targets | ||
+ | |||
+ | Ab Linux 7 werden die Targets mit " | ||
+ | |||
+ | |||
+ | ==Notwendige Pakete einspielen== | ||
+ | <code bash> | ||
+ | yum install targetd | ||
+ | yum install targetcli | ||
+ | </ | ||
+ | |||
+ | Um das folgende spätere Problem beim Starten des „targetd“ zu vermeiden, ein Password in der targetd.yaml hinterlegen: | ||
+ | < | ||
+ | #Probem | ||
+ | | ||
+ | |||
+ | vi / | ||
+ | |||
+ | .. | ||
+ | vi / | ||
+ | |||
+ | #Password eintragen | ||
+ | password: | ||
+ | |||
+ | #vg eintragen | ||
+ | pool_name: | ||
+ | |||
+ | user: root | ||
+ | ssl: false | ||
+ | |||
+ | #Target Namen dann später in der Konfiguration verwenden! | ||
+ | target_name: | ||
+ | |||
+ | |||
+ | </ | ||
+ | <fc # | ||
+ | |||
+ | ==Vorhandene Cluster Platten aus dem LVM anzeigen lassen== | ||
+ | <code bash> | ||
+ | lvs | ||
+ | LV VG | ||
+ | .. | ||
+ | acfs01 | ||
+ | data01 | ||
+ | data02 | ||
+ | data03 | ||
+ | redoa01 racstore01 -wi-a----- 10.00g | ||
+ | redob01 racstore01 -wi-a----- 10.00g | ||
+ | vota01 | ||
+ | votb01 | ||
+ | votc01 | ||
+ | |||
+ | </ | ||
+ | |||
+ | ===Konfiguration erstellen=== | ||
+ | Die Konfiguraton wird mit dem „targetcli“ Utility erstellt. | ||
+ | ==Zuvor evtl. alten targetcli Konfigurationen löschen== | ||
+ | <code bash> | ||
+ | targetcli clearconfig confirm=true | ||
+ | All configuration cleared | ||
+ | </ | ||
+ | |||
+ | |||
+ | ==targetcli aufrufen== | ||
+ | <code bash> | ||
+ | targetcli | ||
+ | </ | ||
+ | |||
+ | Um Fehlermeldung beim Anlegen zu vermeiden, folgende Eigenschaft setzen: | ||
+ | |||
+ | targetcli: | ||
+ | <code bash> | ||
+ | cd / | ||
+ | /> set global export_backstore_name_as_model=false | ||
+ | </ | ||
+ | |||
+ | |||
+ | ==Die vorbereiteten LV's als " | ||
+ | |||
+ | targetcli: | ||
+ | <code bash> | ||
+ | |||
+ | /> cd backstores/ | ||
+ | |||
+ | create rac01-vota01 / | ||
+ | create | ||
+ | create | ||
+ | create | ||
+ | create | ||
+ | create | ||
+ | create | ||
+ | create | ||
+ | create | ||
+ | |||
+ | |||
+ | / | ||
+ | |||
+ | o- block ...................................................................................................... [Storage Objects: 9] | ||
+ | o- rac01-acfs01 ........................................................ [/ | ||
+ | o- rac01-data01 ........................................................ [/ | ||
+ | o- rac01-data02 ........................................................ [/ | ||
+ | o- rac01-data03 ........................................................ [/ | ||
+ | o- rac01-redoa01 ...................................................... [/ | ||
+ | o- rac01-redob01 ...................................................... [/ | ||
+ | o- rac01-vota01 ......................................................... [/ | ||
+ | o- rac01-votb01 ......................................................... [/ | ||
+ | o- rac01-votc01 ......................................................... [/ | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | ==IQN / WWN des Server hinterlegen== | ||
+ | |||
+ | targetcli: | ||
+ | <code bash> | ||
+ | |||
+ | cd / | ||
+ | cd /iscsi | ||
+ | |||
+ | create iqn.2015-03.pipperr.local: | ||
+ | |||
+ | Created target iqn.2015-03.pipperr.local: | ||
+ | Created TPG 1. | ||
+ | |||
+ | ls | ||
+ | |||
+ | </ | ||
+ | |||
+ | ==ACL' | ||
+ | |||
+ | targetcli: | ||
+ | <code bash> | ||
+ | |||
+ | cd / | ||
+ | |||
+ | / | ||
+ | |||
+ | create iqn.2015-03.pipperr.local: | ||
+ | create iqn.2015-03.pipperr.local: | ||
+ | create iqn.2015-03.pipperr.local: | ||
+ | |||
+ | |||
+ | ls | ||
+ | |||
+ | o- acls .................................................................................................................. [ACLs: 3] | ||
+ | o- iqn.2015-03.pipperr.local: | ||
+ | o- iqn.2015-03.pipperr.local: | ||
+ | o- iqn.2015-03.pipperr.local: | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | ==LUN' | ||
+ | |||
+ | targetcli: | ||
+ | <code bash> | ||
+ | |||
+ | cd / | ||
+ | |||
+ | create / | ||
+ | create / | ||
+ | create / | ||
+ | create / | ||
+ | create / | ||
+ | create / | ||
+ | create / | ||
+ | create / | ||
+ | create / | ||
+ | |||
+ | |||
+ | ls | ||
+ | |||
+ | o- luns .................................................................................................................. [LUNs: 9] | ||
+ | o- lun0 ............................................................................ [block/ | ||
+ | o- lun1 ............................................................................ [block/ | ||
+ | o- lun2 ............................................................................ [block/ | ||
+ | o- lun3 ............................................................................ [block/ | ||
+ | o- lun4 .......................................................................... [block/ | ||
+ | o- lun5 .......................................................................... [block/ | ||
+ | o- lun6 ............................................................................ [block/ | ||
+ | o- lun7 ............................................................................ [block/ | ||
+ | o- lun8 ............................................................................ [block/ | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | ==Portal anlegen== | ||
+ | |||
+ | targetcli: | ||
+ | <code bash> | ||
+ | cd / | ||
+ | |||
+ | / | ||
+ | |||
+ | Using default IP port 3260 | ||
+ | Binding to INADDR_ANY (0.0.0.0) | ||
+ | Created network portal 0.0.0.0: | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | ==Konfiguration überprüfen und speichern== | ||
+ | |||
+ | targetcli: | ||
+ | <code bash> | ||
+ | |||
+ | cd / | ||
+ | |||
+ | ls | ||
+ | |||
+ | |||
+ | saveconfig | ||
+ | Last 10 configs saved in / | ||
+ | Configuration saved to / | ||
+ | |||
+ | exit | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | ==Den targetd service aktiveren== | ||
+ | |||
+ | <code bash> | ||
+ | systemctl enable targetd | ||
+ | systemctl status targetd | ||
+ | </ | ||
+ | |||
+ | |||
+ | === Testen auf dem lokalen Client=== | ||
+ | |||
+ | Mit den iSCSI Client auf der lokalen Maschine überprüfen. | ||
+ | |||
+ | |||
+ | <code bash> | ||
+ | # Client installieren | ||
+ | yum install iscsi-initiator-utils | ||
+ | # Client configurieren | ||
+ | cd /etc/iscsi/ | ||
+ | vi initiatorname.iscsi | ||
+ | InitiatorName=iqn.2015-03.pipperr.local: | ||
+ | |||
+ | #scsi service neu starten | ||
+ | systemctl restart iscsid | ||
+ | systemctl restart iscsi | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | Testen mit: | ||
+ | <code bash> | ||
+ | |||
+ | iscsiadm -m discovery -t sendtargets -p storage01 | ||
+ | 10.10.10.180: | ||
+ | |||
+ | iscsiadm --mode node --targetname iqn.2015-03.pipperr.local: | ||
+ | |||
+ | #alle Block Devices anzeigen lassen, die neuen Platten sollten am Ende angezeigt werden | ||
+ | lsblk --scsi | ||
+ | |||
+ | #prüfen das die Platte nicht auf Readonly (RM=1) steht! | ||
+ | lsblk | egrep " | ||
+ | NAME | ||
+ | sdk 8:160 0 2G 0 disk | ||
+ | |||
+ | |||
+ | |||
+ | #Meldung ausgeben lassen das neue Platten gebunden wurden | ||
+ | dmesg | ||
+ | |||
+ | |||
+ | #Neue Devices anzeigen lassen | ||
+ | fdisk -l | grep Disk | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | Quellen: | ||
+ | * http:// | ||
+ | * http:// | ||
+ | * https:// | ||
+ | * https:// | ||
+ | |||
+ | |||
+ | ---- | ||
+ | |||
+ | |||
+ | ==== Bereitstellen der iSCSI Lun auf dem Zielsystem ==== | ||
+ | |||
+ | Zielsystem : Oracle Linux 7.0 64bit | ||
+ | |||
+ | |||
+ | == Namensauflösung für iSCSI Server prüfen oder statisch einrichten == | ||
+ | |||
+ | In Host Datei eintragen: | ||
+ | <code bash> | ||
+ | vi /etc/hosts | ||
+ | .. | ||
+ | 10.10.10.180 storage01.pipperr.local storage01 | ||
+ | </ | ||
+ | |||
+ | ==Paket "iSCSI -initiator-utils" | ||
+ | <code bash> | ||
+ | yum install iscsi-initiator-utils | ||
+ | |||
+ | systemctl enable iscsid | ||
+ | systemctl enable iscsi | ||
+ | |||
+ | </ | ||
+ | |||
+ | == InitiatorName anpassen == | ||
+ | |||
+ | <code bash> | ||
+ | cd /etc/iscsi | ||
+ | |||
+ | #Namen anpassen! | ||
+ | vi initiatorname.iscsi | ||
+ | |||
+ | InitiatorName=iqn.2015-03.pipperr.local: | ||
+ | |||
+ | </ | ||
+ | |||
+ | < | ||
+ | |||
+ | |||
+ | |||
+ | ==Konfigurieren der Luns über die Datei iscsid.conf== | ||
+ | <code bash> | ||
+ | vi / | ||
+ | .. | ||
+ | node.startup = automatic | ||
+ | .. | ||
+ | </ | ||
+ | |||
+ | |||
+ | ==Starten== | ||
+ | <code bash> | ||
+ | |||
+ | #scsi service neu starten | ||
+ | systemctl restart iscsid | ||
+ | systemctl restart iscsi | ||
+ | |||
+ | </ | ||
+ | |||
+ | ==Targets erkennen== | ||
+ | <code bash> | ||
+ | # netz testen | ||
+ | ping storage01 | ||
+ | |||
+ | # Was stellt der Server zur Verfügung | ||
+ | iscsiadm -m discovery -t sendtargets -p storage01 | ||
+ | |||
+ | 10.10.10.180: | ||
+ | |||
+ | # | ||
+ | |||
+ | iscsiadm -m node -o show | ||
+ | |||
+ | </ | ||
+ | |||
+ | ==Anmelden== | ||
+ | <code bash> | ||
+ | iscsiadm -m node --login | ||
+ | |||
+ | Logging in to [iface: default, target: iqn.2015-03.pipperr.local: | ||
+ | Login to [iface: default, target: iqn.2015-03.pipperr.local: | ||
+ | |||
+ | </ | ||
+ | |||
+ | Session anzeigen: | ||
+ | <code bash> | ||
+ | iscsiadm -m session -P 3 | ||
+ | </ | ||
+ | |||
+ | ==Überwachen== | ||
+ | <code bash> | ||
+ | iscsiadm -m session -o show | ||
+ | </ | ||
+ | |||
+ | |||
+ | == Platten anzeigen lassen == | ||
+ | |||
+ | < | ||
+ | lsblk --scsi | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | Partition anzeigen lassen: | ||
+ | |||
+ | <code bash> | ||
+ | cat / | ||
+ | ... | ||
+ | | ||
+ | </ | ||
+ | |||
+ | |||
+ | ==Disk wieder abmelden== | ||
+ | |||
+ | <code bash> | ||
+ | iscsiadm -m node -T iqn.2015-03.pipperr.local: | ||
+ | </ | ||
+ | |||
+ | |||
+ | ==Automatisch anmelden== | ||
+ | <code bash> | ||
+ | iscsiadm -m node -T iqn.2015-03.pipperr.local: | ||
+ | </ | ||
+ | |||
+ | |||
+ | bzw: mit " | ||
+ | |||
+ | |||
+ | |||
+ | Booten und testen ob die Platten erneut im System angezeigt werden. | ||
+ | |||
+ | |||
+ | ---- | ||
+ | |||
+ | |||
+ | |||
+ | ---- | ||
+ | |||
+ | ====Auf dem RAC Knoten ASM Platte einrichten==== | ||
+ | |||
+ | == ASM Library installieren == | ||
+ | <code bash> | ||
+ | yum install oracleasm-support | ||
+ | </ | ||
+ | |||
+ | ==Oracle User anlegen== | ||
+ | |||
+ | <code bash> | ||
+ | |||
+ | groupadd -g 1000 oinstall | ||
+ | groupadd -g 1001 dba | ||
+ | groupadd -g 1002 asmadmin | ||
+ | |||
+ | useradd -u 1100 -g oinstall -G dba oracle | ||
+ | usermod -a -G asmadmin oracle | ||
+ | passwd oracle | ||
+ | |||
+ | |||
+ | useradd -u 1101 -g oinstall -G dba grid | ||
+ | usermod -a -G asmadmin grid | ||
+ | passwd grid | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | == ASM konfigurieren auf allen Knoten == | ||
+ | |||
+ | **Als user root!** | ||
+ | |||
+ | Damit der Grid User UND der Oracle User auf die Platten zugreifen, die Gruppe asmadmin | ||
+ | |||
+ | <code bash> | ||
+ | |||
+ | oracleasm configure -i | ||
+ | |||
+ | Default user to own the driver interface []: grid | ||
+ | Default group to own the driver interface []: asmadmin | ||
+ | Scan for Oracle ASM disks on boot (y/n) [y]: y | ||
+ | Writing Oracle ASM library driver configuration: | ||
+ | |||
+ | </ | ||
+ | |||
+ | == ASM starten == | ||
+ | |||
+ | **Als user root!** | ||
+ | <code bash> | ||
+ | oracleasm init | ||
+ | </ | ||
+ | |||
+ | == Platten partitionieren == | ||
+ | |||
+ | < | ||
+ | |||
+ | lsblk | ||
+ | |||
+ | |||
+ | # neue primäre Partition über die ganze Platte anlegen | ||
+ | fdisk /dev/sdb | ||
+ | |||
+ | #In each case, the sequence of answers is | ||
+ | |||
+ | " | ||
+ | |||
+ | </ | ||
+ | |||
+ | ==ASM Header auf die Platten schreiben schreiben== | ||
+ | |||
+ | < | ||
+ | |||
+ | #Jede Platten erzeugen | ||
+ | oracleasm createdisk DATA01 /dev/sdb1 | ||
+ | |||
+ | Writing disk header: done | ||
+ | Instantiating disk: done | ||
+ | |||
+ | |||
+ | # Platten einlesen | ||
+ | |||
+ | oracleasm listdisks | ||
+ | |||
+ | </ | ||
+ | |||
+ | == Reboot Test == | ||
+ | |||
+ | Reboot und testen ob die Platten alle wieder da sind | ||
+ | |||
+ | <code bash> | ||
+ | oracleasm listdisks | ||
+ | </ | ||
+ | |||
+ | Werden keine Platten angezeigt neu scannen: | ||
+ | |||
+ | <code bash> | ||
+ | oracleasm scandisks | ||
+ | |||
+ | Reloading disk partitions: done | ||
+ | Cleaning any stale ASM disks... | ||
+ | Scanning system for ASM disks... | ||
+ | Instantiating disk " | ||
+ | |||
+ | ... | ||
+ | </ | ||
+ | |||
+ | |||
+ | => <fc # | ||
+ | |||
+ | Der ASM Service/ | ||
+ | |||
+ | Einrichten der Abhängigkeiten über die Service Eigenschaften " | ||
+ | <code bash> | ||
+ | |||
+ | vi / | ||
+ | |||
+ | [Unit] | ||
+ | Description=Load oracleasm Modules | ||
+ | |||
+ | # | ||
+ | |||
+ | equires=iscsi.service | ||
+ | After=iscsi.service | ||
+ | ... | ||
+ | |||
+ | #neu starten | ||
+ | |||
+ | reboot | ||
+ | |||
+ | |||
+ | # | ||
+ | |||
+ | oracleasm listdisks | ||
+ | |||
+ | DATA01 | ||
+ | ... | ||
+ | VOT03 | ||
+ | |||
+ | # | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | Falls Platte gemounted wurde die Eigenschaften mit blkid anzeigen: | ||
+ | <code bash> | ||
+ | blkid | ||
+ | |||
+ | ... | ||
+ | /dev/sdb1: LABEL=" | ||
+ | ... | ||
+ | /dev/sdj1: LABEL=" | ||
+ | |||
+ | # | ||
+ | |||
+ | blkid -i /dev/sdj1 | ||
+ | |||
+ | DEVNAME=/ | ||
+ | MINIMUM_IO_SIZE=512 | ||
+ | OPTIMAL_IO_SIZE=4194304 | ||
+ | PHYSICAL_SECTOR_SIZE=512 | ||
+ | LOGICAL_SECTOR_SIZE=512 | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | ---- | ||
+ | |||
+ | ==== Last Test ==== | ||
+ | |||
+ | * Orion | ||
+ | * => http:// | ||
+ | * => http:// | ||
+ | |||
+ | * slob => http:// | ||
+ | |||
+ | |||
+ | === Orion === | ||
+ | |||
+ | Nur noch über die Japanische Oracle Website herunterladbar, | ||
+ | |||
+ | <code bash> | ||
+ | |||
+ | yum install libaio | ||
+ | |||
+ | gzip -d orion_linux_x86-64.gz | ||
+ | |||
+ | chmod 764 orion_linux_x86-64 | ||
+ | |||
+ | ./ | ||
+ | ORION: ORacle IO Numbers -- Version 11.1.0.7.0 | ||
+ | |||
+ | |||
+ | #erste Test | ||
+ | |||
+ | #Platte angeben über eine Steuerdatei mit dem Name des Testes | ||
+ | |||
+ | vi iSCSI.lun | ||
+ | / | ||
+ | |||
+ | |||
+ | #starten | ||
+ | ./ | ||
+ | |||
+ | #Ergebnis | ||
+ | # ein meiner VMware Testumgebung kann ich ~56MB/s erreichen | ||
+ | |||
+ | #Ergebnis anzeigen: | ||
+ | |||
+ | cat iSCSI_20150309_2350_summary.txt | ||
+ | |||
+ | Maximum Large MBPS=55.99 | ||
+ | Maximum Small IOPS=3290 | ||
+ | Minimum Small Latency=0.56 @ Small=1 and Large=0 | ||
+ | |||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | siehe auch [[dba: | ||
+ | ---- | ||
+ | |||
+ | |||
+ | |||
+ | ==== Problem beim Start von targetd ==== | ||
+ | |||
+ | <fc # | ||
+ | |||
+ | <code bash> | ||
+ | |||
+ | [root@storage01 target]# systemctl stop targetd | ||
+ | [root@storage01 target]# systemctl start targetd | ||
+ | [root@storage01 target]# systemctl status targetd -l | ||
+ | targetd.service - targetd storage array API daemon | ||
+ | | ||
+ | | ||
+ | Process: 2231 ExecStart=/ | ||
+ | Main PID: 2231 (code=exited, | ||
+ | |||
+ | Mar 08 23:24:30 storage01.pipperr.local targetd[2231]: | ||
+ | Mar 08 23:24:30 storage01.pipperr.local targetd[2231]: | ||
+ | Mar 08 23:24:30 storage01.pipperr.local targetd[2231]: | ||
+ | Mar 08 23:24:30 storage01.pipperr.local targetd[2231]: | ||
+ | Mar 08 23:24:30 storage01.pipperr.local targetd[2231]: | ||
+ | Mar 08 23:24:30 storage01.pipperr.local targetd[2231]: | ||
+ | Mar 08 23:24:30 storage01.pipperr.local targetd[2231]: | ||
+ | Mar 08 23:24:30 storage01.pipperr.local targetd[2231]: | ||
+ | Mar 08 23:24:30 storage01.pipperr.local systemd[1]: targetd.service: | ||
+ | Mar 08 23:24:30 storage01.pipperr.local systemd[1]: Unit targetd.service entered failed state. | ||
+ | |||
+ | </ | ||
+ | |||
+ | Fehlermeldung ist wohl mehr als nichtssagend..... | ||
+ | |||
+ | <fc # | ||
+ | |||
+ | In der targetd.yaml ist das Password falsch angegeben, in diesen YAML Dateien muss ein Leerzeichen nach dem ":" | ||
+ | |||
+ | < | ||
+ | vi / | ||
+ | |||
+ | |||
+ | password: abcde1234! | ||
+ | # defaults below; uncomment and edit | ||
+ | pool_name: racstore01 | ||
+ | user: root | ||
+ | ssl: false | ||
+ | target_name: | ||
+ | |||
+ | </ | ||
+ | |||
+ | Auch müssen die anderen Werte gesetzt werden, sonst folgen weiter Fehlermeldungen dieser Art! | ||
+ | |||
+ | |||
+ | ---- | ||
+ | |||
+ | === ASM Platten löschen und neu anlegen === | ||
+ | |||
+ | Muss eine ASM Platte neu initialisiert werden:, zum Beispiel nach einer fehlgeschlagenen Installation: | ||
+ | |||
+ | Auf ersten Knoten: | ||
+ | <code bash> | ||
+ | oracleasm deletedisk VOT01 | ||
+ | oracleasm deletedisk VOT02 | ||
+ | oracleasm deletedisk VOT03 | ||
+ | |||
+ | </ | ||
+ | |||
+ | Auf zweiten Knoten neu alles einlesen | ||
+ | <code bash> | ||
+ | oracleasm scandisks | ||
+ | oracleasm listdisks | ||
+ | </ | ||
+ | |||
+ | Platten neu anlegen auf dem Knoten 1 | ||
+ | <code bash> | ||
+ | lsblk | ||
+ | oracleasm createdisk VOT1 /dev/sdh1 | ||
+ | oracleasm createdisk VOT2 /dev/sdi1 | ||
+ | oracleasm createdisk VOT3 /dev/sdj1 | ||
+ | |||
+ | oracleasm listdisks | ||
+ | |||
+ | </ | ||
+ | |||
+ | Auf dem Knoten 2 neu einlesen: | ||
+ | <code bash> | ||
+ | oracleasm scandisks | ||
+ | oracleasm listdisks | ||
+ | </ | ||
+ | |||
+ | |||
+ | ---- | ||
+ | |||
+ | === ASM Platten Größen verändern === | ||
+ | |||
+ | Bei der Installation hat sich herausgestellt, | ||
+ | |||
+ | RAC Umgebung daher gestoppt und Plattengröße angepasst, <fc # | ||
+ | |||
+ | |||
+ | Wieviel Platz hat das Volume noch: | ||
+ | <code bash> | ||
+ | pvs /dev/sdb | ||
+ | PV | ||
+ | / | ||
+ | |||
+ | </ | ||
+ | => 34GB Frei | ||
+ | |||
+ | Was belegt den Platz: | ||
+ | <code bash> | ||
+ | lvs racstore01 | ||
+ | LV VG | ||
+ | acfs01 | ||
+ | data01 | ||
+ | data02 | ||
+ | data03 | ||
+ | redoa01 racstore01 -wi-ao---- 10.00g | ||
+ | redob01 racstore01 -wi-ao---- 10.00g | ||
+ | vota01 | ||
+ | votb01 | ||
+ | votc01 | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | Voting Platten auf 6 GB für 12c erweiteren: | ||
+ | <code bash> | ||
+ | lvextend -L6G / | ||
+ | lvextend -L6G / | ||
+ | lvextend -L6G / | ||
+ | |||
+ | #prüfen | ||
+ | |||
+ | lvs | grep vot | ||
+ | vota01 | ||
+ | votb01 | ||
+ | votc01 | ||
+ | |||
+ | </ | ||
+ | |||
+ | Akuelle Größe der Partition bestimmen und neu anlegen zum Vergrößern | ||
+ | |||
+ | <code bash> | ||
+ | |||
+ | #anzeigen | ||
+ | fdisk / | ||
+ | |||
+ | #löschen und neu anlegen | ||
+ | fdisk /dev/sda1 | ||
+ | |||
+ | </ | ||
+ | Anzeigen lassen mit p, löschen mit d , Anzeige mit p, mit n neu anlegen (als wie zuvor als Primary) mit p testen und mit w zum Schluss schreiben. | ||
+ | |||
+ | Reboot, damit die Platten neu eingelesen werden. | ||
+ | |||
+ | == Rac knoten 1 == | ||
+ | Rac Knoten starten und dort die ASM Platte neu anlegen: | ||
+ | <code bash> | ||
+ | oracleasm createdisk VOT01 /dev/sdh1 | ||
+ | oracleasm createdisk VOT02 /dev/sdi1 | ||
+ | oracleasm createdisk VOT03 /dev/sdj1 | ||
+ | </ | ||
+ | |||
+ | <fc # | ||
+ | |||
+ | |||
+ | ---- | ||
+ | |||
+ | ==== ASM Platten hinzufügen ==== | ||
+ | |||
+ | |||
+ | Wie groß ist unser Volumen: | ||
+ | |||
+ | <code bash> | ||
+ | |||
+ | vgdisplay racstore01 | ||
+ | |||
+ | .. | ||
+ | VG Size | ||
+ | .. | ||
+ | Free PE / Size 2507 / 9.79 GiB | ||
+ | .. | ||
+ | |||
+ | #Oder | ||
+ | pvs /dev/sdb | ||
+ | PV | ||
+ | / | ||
+ | |||
+ | |||
+ | #Wieviel Platz hat die Physik darunter noch | ||
+ | |||
+ | fdisk / | ||
+ | p | ||
+ | .. | ||
+ | Disk /dev/sdb: 150.3 GB, 150323855360 bytes, 293601280 sectors | ||
+ | ... | ||
+ | |||
+ | </ | ||
+ | |||
+ | Wir möchten eine weitere 20GB Lun anlegen, d.h. wir benötigen min. 12GB. | ||
+ | |||
+ | Das klappt nicht, wir brauchen daher eine neue Platte | ||
+ | |||
+ | == Disk hinzufügen== | ||
+ | |||
+ | Disk pyhsikalisch hinzufügen ( VMWare Neue Platte einbinden) | ||
+ | |||
+ | == Disk im OS erkennen== | ||
+ | |||
+ | rescan initialiseren | ||
+ | <code bash> | ||
+ | grep mpt / | ||
+ | sys/ | ||
+ | |||
+ | echo "- - -" > / | ||
+ | |||
+ | tail / | ||
+ | |||
+ | Jun 2 14:39:13 storage01 kernel: sdn: unknown partition table | ||
+ | Jun 2 14:39:13 storage01 kernel: sd 2:0:2:0: [sdn] Cache data unavailable | ||
+ | Jun 2 14:39:13 storage01 kernel: sd 2:0:2:0: [sdn] Assuming drive cache: write through | ||
+ | Jun 2 14:39:13 storage01 kernel: sd 2:0:2:0: [sdn] Attached SCSI disk | ||
+ | |||
+ | |||
+ | fdisk -l | ||
+ | |||
+ | ... | ||
+ | Disk /dev/sdn: 150.3 GB, 150323855360 bytes, 293601280 sectors | ||
+ | ... | ||
+ | </ | ||
+ | |||
+ | == Disk partionieren== | ||
+ | |||
+ | <code bash> | ||
+ | Command (m for help): p | ||
+ | |||
+ | Disk /dev/sdn: 150.3 GB, 150323855360 bytes, 293601280 sectors | ||
+ | Units = sectors of 1 * 512 = 512 bytes | ||
+ | Sector size (logical/ | ||
+ | I/O size (minimum/ | ||
+ | Disk label type: dos | ||
+ | Disk identifier: 0x6e96c8e8 | ||
+ | |||
+ | | ||
+ | |||
+ | Command (m for help): n | ||
+ | Partition type: | ||
+ | | ||
+ | | ||
+ | Select (default p): p | ||
+ | Partition number (1-4, default 1): 1 | ||
+ | First sector (2048-293601279, | ||
+ | Using default value 2048 | ||
+ | Last sector, +sectors or +size{K, | ||
+ | Using default value 293601279 | ||
+ | Partition 1 of type Linux and of size 140 GiB is set | ||
+ | |||
+ | Command (m for help): t | ||
+ | Selected partition 1 | ||
+ | Hex code (type L to list all codes): 8e | ||
+ | Changed type of partition ' | ||
+ | |||
+ | Command (m for help): w | ||
+ | The partition table has been altered! | ||
+ | |||
+ | Calling ioctl() to re-read partition table. | ||
+ | Syncing disks. | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | == Disk der VG hinzufügen== | ||
+ | <code bash> | ||
+ | |||
+ | pvcreate /dev/sdn1 | ||
+ | Physical volume "/ | ||
+ | |||
+ | |||
+ | vgextend racstore01 /dev/sdn1 | ||
+ | Volume group " | ||
+ | |||
+ | pvscan | ||
+ | PV / | ||
+ | PV / | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | == Neue ASM Disks erzeugen == | ||
+ | |||
+ | <code bash> | ||
+ | |||
+ | lvcreate --size 20G -n data04 racstore01 | ||
+ | Logical volume " | ||
+ | |||
+ | lvcreate --size 20G -n data05 racstore01 | ||
+ | Logical volume " | ||
+ | |||
+ | cd / | ||
+ | |||
+ | ls -l data* | ||
+ | lrwxrwxrwx 1 root root 7 Mar 13 15:01 data01 -> ../dm-7 | ||
+ | lrwxrwxrwx 1 root root 7 Mar 13 15:01 data02 -> ../dm-8 | ||
+ | lrwxrwxrwx 1 root root 7 Mar 13 15:01 data03 -> ../dm-9 | ||
+ | lrwxrwxrwx 1 root root 8 Jun 2 14:50 data04 -> ../dm-13 | ||
+ | lrwxrwxrwx 1 root root 8 Jun 2 14:50 data05 -> ../dm-14 | ||
+ | |||
+ | </ | ||
+ | |||
+ | == Als Traget zur Verfügung stellen== | ||
+ | |||
+ | <code bash> | ||
+ | |||
+ | targetcli | ||
+ | targetcli shell version 2.1.fb41 | ||
+ | Copyright 2011-2013 by Datera, Inc and others. | ||
+ | For help on commands, type ' | ||
+ | |||
+ | / | ||
+ | /> set global export_backstore_name_as_model=false | ||
+ | Parameter export_backstore_name_as_model is now ' | ||
+ | /> cd backstores/ | ||
+ | / | ||
+ | |||
+ | / | ||
+ | Created block storage object rac01-data04 using / | ||
+ | / | ||
+ | Created block storage object rac01-data05 using / | ||
+ | / | ||
+ | ... | ||
+ | o- rac01-data04 ........................................................ [/ | ||
+ | o- rac01-data05 ........................................................ [/ | ||
+ | | ||
+ | / | ||
+ | |||
+ | / | ||
+ | Created LUN 11. | ||
+ | Created LUN 11->11 mapping in node ACL iqn.2015-03.pipperr.local: | ||
+ | Created LUN 11->11 mapping in node ACL iqn.2015-03.pipperr.local: | ||
+ | Created LUN 11->11 mapping in node ACL iqn.2015-03.pipperr.local: | ||
+ | / | ||
+ | Created LUN 12. | ||
+ | Created LUN 12->12 mapping in node ACL iqn.2015-03.pipperr.local: | ||
+ | Created LUN 12->12 mapping in node ACL iqn.2015-03.pipperr.local: | ||
+ | Created LUN 12->12 mapping in node ACL iqn.2015-03.pipperr.local: | ||
+ | / | ||
+ | /> | ||
+ | /> saveconfig | ||
+ | Last 10 configs saved in / | ||
+ | Configuration saved to / | ||
+ | /> exit | ||
+ | Global pref auto_save_on_exit=true | ||
+ | Last 10 configs saved in / | ||
+ | Configuration saved to / | ||
+ | |||
+ | </ | ||
+ | |||
+ | == Platten auf dem RAC Knoten neu erkennen== | ||
+ | |||
+ | Am einfachsten beide Knoten neu starten, falls möglich, ansonsten online neu erkennen lassen | ||
+ | |||
+ | |||
+ | Das schwierige wird nun über die ganze device Nummer die richtigen Platten auf dem Server zu erkennen .... | ||
+ | |||
+ | Auf dem ISCI Server das mapping abfragen | ||
+ | |||
+ | <code bash> | ||
+ | targetcli ls | ||
+ | |||
+ | ... | ||
+ | | ||
+ | .. | ||
+ | o- mapped_lun9 .......................................................................... [lun9 block/ | ||
+ | | | ||
+ | |||
+ | ... | ||
+ | |||
+ | </ | ||
+ | |||
+ | Auf dem Rac Knoten 1 | ||
+ | <code bash> | ||
+ | cd / | ||
+ | |||
+ | ls -la | grep iqn.2016-09.pipperr.local | grep lun-9 | ||
+ | lrwxrwxrwx 1 root root 9 Jun 2 15:16 ip-10.10.10.182: | ||
+ | |||
+ | ls -la | grep iqn.2016-09.pipperr.local | grep lun-10 | ||
+ | lrwxrwxrwx 1 root root 9 Jun 2 15:16 ip-10.10.10.182: | ||
+ | |||
+ | lsblk /dev/sdw | ||
+ | NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT | ||
+ | sdw | ||
+ | |||
+ | lsblk /dev/sdu | ||
+ | NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT | ||
+ | sdu | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | so können wir erkennen das wir für die ASM Platte DATA04_02 das Device /dev/sdu verwenden müssen und für DATA05_02 das Device /dev/sdw | ||
+ | |||
+ | Auf beiden mit fdisk eine Partition anlegen, mit oracleasm stemplen und auf Knoten 2 mit oracleasm scandisk einbinden. | ||
+ | |||
+ | Das gleiche für das zweite ISCSI Storage, verwende hier eine mirror über zwei RZ Umgebungen mit Oracle ASM. | ||
+ | |||
+ | |||
+ | ASM Platten dann einbinden mit: | ||
+ | <code sql> | ||
+ | # als grid mit sqlplus / as sysasm | ||
+ | |||
+ | alter diskgroup data add | ||
+ | failgroup STORAGE1 disk '/ | ||
+ | failgroup STORAGE2 disk '/ | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | ---- | ||
+ | |||
vmware/iscsi_target_for_shared_disks.txt · Zuletzt geändert: 2018/06/02 18:21 von gpipperr