CSE Support Wiki

User Tools

Site Tools

Trace: • backup
admin:procedures

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
admin:procedures [2019/07/05 15:36]
peppe [Table] 		admin:procedures [2022/11/09 09:45] (current)
peppe
Line 10: Line 10:
 === uranus === === uranus ===
   # accmake.sh <user>   # accmake.sh <user>
 +Δημιουργία e-mail .forward για προπτυχιακούς 2019-20 και μετά.
  
 === gaia === === gaia ===
Line 15: Line 16:
   # newaliases   # newaliases
   # setstats_student.sh <user>   # setstats_student.sh <user>
 +  # webmail_welcome.sh <user> (για gcloud redirection)
  
 === titan-new (10.7.4.44) === === titan-new (10.7.4.44) ===
Line 41: Line 43:
 ---- ----
  
 +==== Τροποποίηση ====
 +
 +=== uranus ===
 +  # user home rename
 +  # emailforward-single.sh <newuser>
 +
 +=== gaia ===
 +  # vi /etc/aliases (μετονομασία χρήστη)
 +  # newaliases
 +  # mv /var/mail/<olduser> /var/mail/<newuser>
 +  # forward_mbox.sh <newuser>
 +
 +=== titan-new (10.7.4.44) ===
 +  # pkusers --delete <olduser>
 +  # ./bin/create_student_quotas.sh <newuser>
 +
 +=== scylla ===
 +  # cd /var/yp
 +  # vi master.passwd (μετονομασία χρήστη/αποθήκευση)
 +  # make
 +
 +----
 +==== Μαζική δημιουργία ====
 +
 +=== e-mail forwarding @uoi.gr ===
 +Τοποθέτηση readonly .forward σε κάθε λογαριασμό με προώθηση στο cs{0x5AM}@uoi.gr
 +
 +  for acc in cse9[0-9]*; do echo "cs0"`echo $acc | cut -b 5-8`"@uoi.gr" > ${acc}/.forward; done
 +
 +----
 ==== Αντίγραφο ασφαλείας ==== ==== Αντίγραφο ασφαλείας ====
  
Line 53: Line 85:
  
  
 +----
 ==== Πρόβλημα "error in service module" κατά την αλλαγή password (scylla) ==== ==== Πρόβλημα "error in service module" κατά την αλλαγή password (scylla) ====
  
   # sh /etc/rc.d/yppasswdd restart   # sh /etc/rc.d/yppasswdd restart
  
 +
 +----
 +
 +==== Λογαριασμοί μεταπτυχιακών ====
 +
 +Για μεταγραφή σε λατινικούς χαρακτήρες:\\
 +[[http://www.passport.gov.gr/passports/GrElotConverter/GrElotConverter.html]]\\
 +
 +Αποθήκευση της λίστας σε αρχείο names.txt\\
 +Για παραγωγή usernames:
 +  $ awk '{print tolower(substr($2,1,1)) tolower($1) "\t" $0}' names.txt > grads.txt
 +
 +Παραγωγή αποκόμματος master.passwd.\\
 +uid = πρώτο User ID που θα χρησιμοποιηθεί\\
 +20210922 = ημερομηνία\\
 +21 = ακαδημαϊκό έτος εισαγωγής (π.χ. 21 σημαίνει 2021-22)\\
 +  $ awk -F'\t' 'BEGIN{uid=16368}{printf("%s:*LK-UNUSED-%d*:%d:51::0:0:%s:/usr/home/grads/grads%d/%s:/usr/local/bin/tcsh\n", $1, 20210922, uid, $2, 21, $1); uid++}' grads.txt
 +
 +Εισαγωγή σε scylla:/var/yp/master.passwd (& make)\\
 +\\
 +Δημιουργία homedirs, uranus:
 +  # awk '{print "accmake-grads.sh "$1}' grads.txt
 +Δημιουργία λίστας για gaia:/etc/aliases (& newaliases)
 +  $ awk '{printf("%s, ", $1)}' grads.txt
 +Δημιουργία εντολής για e-mail quotas (copy & paste σε gaia):
 +  $ awk '{print "setstats_grads.sh "$1}' grads.txt
 +Δημιουργία εντολής για print quotas (copy & paste σε titan-new):
 +  $ awk '{print "create_grads_quotas.sh "$1}' grads.txt
  
  
Line 133: Line 193:
  
  
-===== HP StorageWorks MSL2024 Tape Library ===== +===== DRBL - clonezilla server =====
- +
-LTO 3 drive, 400/800GB\\ +
-24 tapes changer +
- +
-==== Reset admin password ==== +
- +
-Default service password: 42311324 +
- +
-new pass: 00000000 +
- +
-==== List SCSI devices ==== +
- +
-  # lsscsi -g +
-  [0:0:0:0]    cd/dvd  TEAC     DW-224E-R        C.AB  /dev/sr0   /dev/sg0  +
-  [2:0:0:0]    disk    ATA      WDC WD1002FBYS-0 0C06  /dev/sda   /dev/sg3  +
-  [3:0:0:0]    disk    ATA      WDC WD1002FBYS-0 0C06  /dev/sdb   /dev/sg4  +
-  [4:0:0:0]    disk    ATA      WDC WD5000AAKS-0 3B01  /dev/sdc   /dev/sg5  +
-  [5:0:0:0]    disk    ATA      WDC WD5000AAKS-0 3B01  /dev/sdd   /dev/sg6  +
-  [7:0:4:0]    tape    HP       Ultrium 3-SCSI   G54W  /dev/st0   /dev/sg1  +
-  [7:0:4:1]    mediumx HP       MSL G3 Series    2.40  /dev/sch0  /dev/sg2 +
- +
-==== Commands ==== +
- +
-tape drive control via mt +
-tape changer control via mtx +
- +
-load tape +
-  # mtx -f /dev/sg2 load <tape #> +
- +
-unload tape (to previous slot) +
-  # mtx -f /dev/sg2 unload +
- +
-check inventory +
-  # mtx -f /dev/sg2 status +
- +
-rewind tape +
-  # mt -f /dev/nst0 rewind +
- +
-dump/restore, mt etc. use /dev/st0 or /dev/nst0 (no rewind)\\  +
-mtx uses raw SCSI device /dev/sg2 +
- +
-write from local file +
-  # dd if=root.fs of=/dev/nst0 bs=4M status=progress +
- +
-==== Media usage ==== +
-{{tablelayout?rowsHeaderSource=Auto&colwidth="72px,111px"}} +
-^  Tape # ^  Media loads  ^ +
-|       1 |               | +
-|       2 |               | +
-|       3 |               | +
-|       4 |               | +
-|       5 |               | +
-|       6 |               | +
-|       7 |               | +
-|       8 |               | +
-|       9 |  116          | +
-|      10 |  102          | +
-|      11 |  72           | +
-|      12 |               | +
-|      13 |               | +
-|      14 |               | +
-|      15 |               | +
-|      16 |               | +
-|      17 |               | +
-|      18 |               | +
-|      19 |               | +
-|      20 |               | +
-|      21 |               | +
-|      22 |               | +
-|      23 |               | +
-|      24 |  Cleaning     | +
- +
-==== Variable block size ==== +
- +
-[[https://unix.stackexchange.com/questions/366039/cannot-allocate-memory-when-reading-from-scsi-tape|stackexchange]] +
- +
-Ok, I think I've worked this out. +
-TL;DR +
- +
-Use dd with a large block size to read from the tape instead: +
- +
-dd if=/dev/nst0 bs=1M | tar tvf - +
- +
-Background +
- +
-When you write to tapes, the data is written in units called blocks. These are like sectors on a hard disk. Where hard disk blocks were fixed at 512-bytes for many years and only recently moved to 4096-byte blocks, tape blocks can be set to any size you like. +
- +
-The block size you wish to use is set with the setblk subcommand in mt-st: +
- +
-mt-st -f /dev/nst0 setblk 512    # Use 512-byte blocks +
-mt-st -f /dev/nst0 setblk 64k    # Use 65536-byte blocks +
- +
-When you issue a read operation to the drive, it will return data in block-sized chunks. You can't read half a block - the smallest amount of data you can read from a tape is one block, which of course could be any number of actual bytes depending on what the block size is. +
- +
-This means if the program you are using supplies a 16kB memory buffer, you will be able to read up to 32 blocks at a time from the tape with 512-byte blocks as these fit exactly in the 16kB buffer. However you will not be able to read anything from the tape with 64kB blocks, because you can't fit even one of them into the 16kB buffer, and remember you can't read anything less than one whole block at a time. +
- +
-Should you attempt to do this, by using a buffer that's too small for one block, the driver (in this case the st SCSI tape driver) will return a memory allocation error code to advise you that your read buffer is too small to hold even a single block.+
  
-To further complicate matters, some tape drives (apparently the LTO ones I am using) also support variable-sized blocks. This means the block size is determined by the size of each write operation and each block can be a different size to the last.+Προσθήκη νέου μηχανήματος
  
-This mode is set with a block size of zero:+==== drbl ====
  
-mt-st -f /dev/nst0 setblk 0    # Use variable-sized blocks+Add details in /etc/dhcp/dhcpd.conf 
 +  # service isc-dhcp-server restart
  
-This is also the default option as - presumably, I am guessing here - it wastes less space with an incorrectly configured program. If, for example, you had set 4k blocks but your program only wrote data in units of 512 bytes at a time, there is a risk that each 512-byte chunk of data would take up 4k on the tape. +run /root/bin/clonezilla-config*.sh
-Cause+
  
-If you now put everything together, you will realise that a tape can hypothetically have a 512-byte block followed by a 64kB block. If the program is reading the tape with a 16kB buffer, it will successfully read the first block, but then when it tries to read more, it won't be able to fit the following 64kB block in its buffer so the driver will return an error.+boot pass: pxelinux
  
-This explains why I was getting Cannot allocate memory errors most of the time, and occasionally I was able to get tar to extract the first few files but then I got the error again. I had not set the block size with mt-st so it had defaulted to variable-sized blocks when the tape was written, and now tar was using too small a buffer to read in some of those blocks.+==== dcs ====
  
-tar has a couple of options for setting its own internal block sizes, namely --blocking-factor, --read-full-records, and --record-size, however these only work if tar is used to directly read and write to the tape.+Change system functionality. Examples in ~/bin
  
-Because I wrote to the tape through the mbuffer program to reduce tape shoe-shining, the block size in the tar archive no longer matched the block size on the tape. This meant --blocking-factor had little effect - it would allow the first block on the tape to be read, which includes a header telling tar what the blocking factor is supposed to be, wherein it switches to that and ignores the value given on the command line. This means the second and subsequent blocks can no longer be read! +unicast: 
-Solution+  drbl-ocs -b -g auto -e1 auto -e2 -x -r -icds -j2 -p reboot -l en_US.UTF-8 startdisk restore ubuntu22.04-hp705ws01-2022-09-28-07-img sda
  
-The solution is to use another program to read from the tape - one that can have the read buffer size set to a value large enough to hold the biggest block we are likely to see.+multicast: 
 +  drbl-ocs -b -g auto -e1 auto -e2 -x -r -icds -j2 -p reboot --clients-to-wait 13 --max-time-to-wait 300 -l en_US.UTF-8 startdisk multicast_restore ubuntu1804LTS-golden-2020-02-11-08-img sda 
 +   
 +===== Labs installation =====
  
-dd works for this, and in a pinch this works:+==== Convert boot from UEFI to legacy (Για ubuntu 18.04 LTS μόνο) ====
  
-dd if=/dev/nst0 bs=256k | tar tvf -+- Boot ubuntu live USB\\ 
 +- Change flag of EFI partition to bios_grub (gparted -> /dev/sda1 -> manage flags -> bios_grub)\\ 
 +  # mount /dev/sda2 /mnt 
 +remove EFI entry: 
 +  # vi /mnt/etc/fstab
  
-You may need to increase 256k if your tape has larger blocks on it, but this worked for me. 1M also works fine so it doesn't appear to matter if the value is too large, within reason.+  # mount -o bind /proc /mnt/proc 
 +  # mount -o bind /dev /mnt/dev 
 +  # mount -o bind /sys /mnt/sys 
 +  # chroot /mnt 
 +  # grub-install /dev/sda
  
admin/procedures.1562330216.txt.gz · Last modified: 2019/07/05 15:36 by peppe

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki