Oborn:

==Network configuration==
Head nodes works as NAT for slave nodes

eth0 connects to outside world
eth1 is internal

Internal network is 10.200.0.0/255.255.0.0

==OS installation==
Normal server Linux install with the following packages:
* ssh server
* tftpd-hpa
* dhcp3-server
* nfs-kernel-server
* debootstrap

* libpmi
* mpich2
* slurm-llnl
* syslinux?

Grab pxelinux from the web

===Setting up a chroot for the node root===
The chroot is a directory on the head nodes that hosts the root filesystem that the slave nodes will use.
A chroot file system is installed with '''debootstrap''' and can be chrooted into so it acts like a standard install.
mkdir -p /nodes/lucid
debootstrap lucid /nodes/lucid/ http://backup.iac.isu.edu:9999/ubuntu/
cp /etc/apt/sources.list /nodes/lucid/etc/apt/
cp /etc/resolv.conf /nodes/lucid/resolv.conf
chroot /nodes/lucid/
apt-get update
locale-gen en_US.UTF-8
dpkg-reconfigure tzdata

More information about [https://wiki.ubuntu.com/DebootstrapChroot chroots], especially helpful locale setup.

In the chroot we want to install the following packages:
* slurm-llnl
* munge
* nfs-common
* openssh-client
* openssh-server

===NFS===

On the head node, add the following to /etc/exports
/nodes/lucid 10.200.0.0/24(ro,async,no_root_squash,no_subtree_check)
/home 10.200.0.0/24(rw,async,root_squash,no_subtree_check)
see '''man 5 exports''' for more information. There are also more nfs settings in

/etc/default/nfs-common
/etc/default/nfs-kernel-server

===Netbooting===

====Setting up dhcp====
Edit /etc/dhcp3/dhcpd.conf as follows:
<pre>
# The ddns-updates-style parameter controls whether or not the server will
# attempt to do a DNS update when a lease is confirmed. We default to the
# behavior of the version 2 packages ('none', since DHCP v2 didn't
# have support for DDNS.)
ddns-update-style none;

# option definitions common to all supported networks...
option domain-name "iac.isu.edu";
#note the comma between dns servers
option domain-name-servers 134.50.254.5, 134.50.57.57;

#these short times are for testing only
default-lease-time 60;
max-lease-time 120;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
authoritative;

# This is a very basic subnet declaration.
subnet 10.0.200.0 netmask 255.255.255.0
{
range 10.0.34.1 10.0.34.200; #we really don't even need a range
option routers 10.0.200.1;
filename "pxelinux.0";
}
#eth0 on brems2
host brems2_0
{
hardware ethernet 00:50:45:5C:10:54;
fixed-address 10.0.200.2;
option host-name "brems2";
}
</pre>

Edit /etc/default/dhcp3-server
INTERFACES=eth1
This will avoid dhcp serving on the outside network!

service dhcp3-server start

====Setting up tftp====
Edit /etc/default/tftpd-hpa
RUN_DAEMON="yes" #had problems with inetd in the past
OPTIONS="-l -a 10.200.0.1 -s /var/lib/tftpboot"

Set up the filesystem to over pxe/tftpd:
mkdir -p /var/lib/tftpboot/pxelinux.cfg
cp /usr/lib/syslinux/pxelinux.0 /var/lib/tftpboot/

Make a file similiar to the following as /var/lib/tftpboot/boot.msg
Booting Brems!!

Put the following in '''/var/lib/tftpboot/pxelinux.cfg/default'''.
The files are referenced relative to /var/lib/tftpboot/
TIMEOUT 5
DISPLAY boot.msg

====Setting up pxelinux====

====Testing====

==Scheduler installation==
The [https://computing.llnl.gov/linux/slurm/quickstart_admin.html Quick Start Administrator Guide] is very helpful.

# Install
* slurm-llnl
* slurm-llnl-slurmdbd
* slurm-llnl-doc
** mkdir /var/run/slurm-llnl

===Munge===
[http://code.google.com/p/munge/ Munge] is an authentication framework recommended by slurm. All the configuration it needs is:
root@brems:# /usr/sbin/create-munge-key
Generating a pseudo-random key using /dev/urandom completed.
root@brems:# /etc/init.d/munge start

==Customising initrd==
Sometimes a customised initrd is necessary.
====Extracting====
Create a directory to extract into:
mkdir init_test; cd init_test
Extract an existing initrd
gzip -d < /var/lib/tftpboot/initrd.img | cpio -io
Edit the file tree as needed. You can add files (modules for instance),
or editing boot script (/init is run by default).

Then package up the directory into a new initrd:
find ./ | cpio -ov | gzip > ../initrd.new

Main Page

2011-02-04T00:29:53Z

Oborn:

MediaWiki:Sitenotice

2011-02-04T00:26:35Z

Oborn: Blanked the page

BackupPC Notes

2011-01-27T00:47:58Z

Oborn: Changed protection level for "BackupPC Notes" ([edit=autoconfirmed] (indefinite) [move=autoconfirmed] (indefinite))

==BackupPC Docs==
*[http://backuppc.sourceforge.net/faq/BackupPC.html BackupPC manual]
*[http://backuppc.wiki.sourceforge.net/ BackupPC wiki]
==Web Interface==
The webpage that hold status messages for BackupPC is [http://192.168.40.196/backuppc/ http://192.168.40.196/backuppc/] The address will change as the system goes live.

==Adding a client for rsync==
===rsync-over-ssh method===
#On the windows box
## Install cygwin
### Make sure to unhide obsolete packages and start from All->uninstall
### admin - cygrunsrv
### base - all
### net - rsync
### net - openssh
## Configure cygwin server: '''ssh-host-config -y'''
### If the config script asks for a password for a cyg_server account, supply one
## Start ssh service: '''cygrunsrv -S sshd'''
## Look for a user to log in as.
### Windows 7 - Admin
### Windows XP - Administrator
### Any version - any other user with admin privileges
## Copy ''id_rsa.pub'' to ''~/.ssh/authorized_keys'' (make sure ~ is the right home dir for the above user)
##Set the firewall
### open port 22 TCP with the scope 134.50.87.0/255.255.255.0,134.50.3.0/255.255.255.0
### In Windows 7, create an exception for the port, then go to properties and change the scope to 134.50.0.0/16
### allow ping
#### (advanced - ICMP in Windows XP)
#### [http://www.sysprobs.com/enable-ping-reply-windows-7 Windows 7] firewall setup for ping
# On vienna
## Log in manually once to accept ssh key and test
## Edit /etc/BackupPC/hosts to add new entry
## In /etc/BackupPC/pc copy to a new file matching name in hosts file
## In the new __.pl file, change
### ''$Conf{ClientNameAlias} = __;'' to match the address
### ''$Conf{XferMethod} = 'rsync'; ''
### '' $Conf{RsyncShareName} = ['/cygdrive/c/']; ''
### '' $Conf{RsyncClientCmd} = '$sshPath -q -x -l iacuser $host $rsyncPath $argList+' '' where ''iacuser'' is the user found above
## Change ownership of new file: '''chown backuppc __.pl'''

===rsyncd method===
# On winnebago (or other Linux box):
## make a new password with makepasswd --chars=128
## update rsyncd.secrets file: echo backuppc:DW045wp1vFY4b22ng53dwbi082Ve8CdsM0m5QpDyEuPvAxLtp2Xc3Q16PwSAwc2XWaj2T5eANN1Yfc1uXvMpoAmmWRV6Wo1FXDuwxMjKQhxhBndh96VI0Np9IWioSYJn > rsyncd.secrets
#On vienna
##edit /etc/BackupPC/hosts to add new entry
##in /etc/BackupPC/pc copy to a new file matching name in hosts file
##run ./checkports.sh to find a free rsync redirection port
##change port, ip, and rsyncpassword in new .pl file
##change ownership of new file: chown backuppc __.pl
#On the client computer:
## Install cygwin w/ rsync on the client with the following packages
### admin - cygrunsrv
### base - all
### net - rsync
### net - openssh
### utils - bzip2
## install rsyncd service: cygrunsrv -I rsyncd -e CYGWIN=nontsec -p c:/cygwin/bin/rsync.exe -a "--daemon --no-detach"
## download rsyncd.conf and rsyncd.secrets file
## start rsyncd service: cygrunsrv -S rsyncd
##Set the firewall
### open port 873 TCP with the scope 134.50.87.0/255.255.255.0,134.50.3.0/255.255.255.0
### allow ping (advanced ICMP)

==Config Files==
Most of the configuration for BackupPC is in '' /etc/backuppc/config.pl'' Many of the options can be over-ridden with host-specific files, but the config.pl should be edited to give a good default for Windows machines. Most importantly ''$Conf{BackupFilesExclude}'' or ''$Conf{RsyncArgs}'' need to be configured to exclude the followng items:
*Windows clients
**'''hiberfil.sys''' holds RAM contents during hibernation
**'''Temporary Internet Files''' IE cache
**Firefox Internet Cache
**'''RECYCLER''' Recyle Bin
**'''pagefile.sys''' Swap file
**'''System Volume Information''' System Restore files
**'''NTUSER.DAT''' Can cause problems with being in use??
**'''TEMP'''
*Linux clients
**'''/temp'''
**'''/dev'''
**'''/proc'''
**'''/sys'''
**'''/mnt'''
**'''/media''' Ubuntu mounts for removable media
**Firefox internet cache
**Other caches??
**Parts of /var??

==Backup Method==
rsync directly over ssh was rejected because of incompatibilities with rsync daemons launched via the ssh process. Instead the preferred method is rsync connected to a persistent rsyncd through an ssh tunnel. This tunnel can either be set up to the machine directly, or through an intermediary such at the IAC nat firewall box. ssh authentication is taken care of through authorized_keys and rsync authentication uses a password that is randomly generated when the client is first set up for backups.

Operating system specific configuration parameters (such as excluded files) are handled in separate files (such as /etc/BackupPC/baseXP.pl) included from host files.

==Storage Setup on Vienna==
Vienna has 6x1TB drives set up under Solaris10 with a ZFS filesystem. The RAID level for the OS is RAID1 and the RAID level for the datapool is RAID10. Compression is disabled in BackupPC and is handled by ZFS instead.

Devices must have at least 1000204885504 bytes. Many 1TB drives do not.

==Notes on booting Vienna==
The hostname isn't set correctly on boot so apache won't start. BackupPC also doesn't start by default.
hostname [-s] vienna
svcadm enable '*apache*'
/etc/init.d/backuppc start
And the webserver is restricted to the local interface, so you need to log in and ssh forward the connection:
ssh vienna.iac.isu.edu -L 8080:localhost:80

==Things we need to think about==
*Open files (Outlook) (shadow drives?)
*Thousands of small files can take very long to index (Pulse Recording can accumulate >60k)
*checksum seeding
*Database dumps from MySQL on webserver
*User notification policy??

User talk:Oborn

2011-01-05T00:58:34Z

Oborn:

2010-12-31T02:24:31Z

Oborn: /* The IAC Wiki has been migrated to a new server. Wiki user accounts now use the IAC Single Sign-On system (used by IAC email, etc). If you do not already have an IAC Single Sign-On username, request an account here. You will need to request the acc

==The IAC Wiki has been migrated to a new server==

Wiki user accounts now use the IAC account system (used by IAC email, brems, mediso, etc).
If you do not already have an IAC account, request an account [https://mail.iac.isu.edu/glados/glados-submitrequest.php here] from an on-campus computer.
Please provide your existing wiki username in the reason section so we can transfer settings.

If you have any problems please contact Brian or Curtis.