Vallard's Tech Notes
Enterprise Datacenter Management Voodoo
updating iDataPlex IMM firmware
Jan 4th
Easiest way is to do it remotely. First, get the package from IBM, then copy the file to some directory. Then do the following:
chmod 755 ibm_fw_imm_yuoo*
./ibm_fw_imm_yuoo48d_linux_32-64.bin -x .
Now you have a bunch of files in this directory. This machine could be a laptop or whatever. Now you can remotely update the IMM as long as you are connected to it through the network.
Do this:
./iflash64 –package imm_yuoo48d.upd –host i01-imm –user xcat –password f00bar
(Here, my host is the IP address of the IMM) Then my userid and password are there. )
The output is as follows:
IBM Command Line IMM Flash Update Utility v1.5
Licensed Materials – Property of IBM
(C) Copyright IBM Corp. 2009 All Rights Reserved.
Connected to IMM at IP address 10.0.0.11.
Update package firmware type: IMM
Update package build level: YUOO48D
Target’s current build level: YUOO24H
Would you like to continue with the update? y/n: y
The IMM is preparing to receive the update.
Transferring image: 100%
Transfer complete.
Validating image.
Updating firmware: 100%
Update complete.
You’ll then need to wait a few seconds for it to become reactivated.
Note: Adding the -s flag will make it so that you are not prompted and works for doing lots of them.
IBM acronyms
Dec 15th
There are many… But these are the ones I hear and need to remember:
DAU – Demand Acceleration Unit (Early release machine that IBM sometimes gives to customers to evaluate.)
BMC – Base Manufacturing Cost (Cost for parts)
TMC – Total Manufacturing Cost (Total cost including labor)
VLH – Vendor Logo Hardware – Hardware that we resell that still has the vendor logo on it.
MES – means an upgrade to a system, but I don’t remember what it stands for.
IPMI Settings
Nov 28th
On a node to set IPMI settings:
ipmitool lan set 1 ipaddr 10.10.0.101
ipmitool lan set 1 netmask 255.255.0.0
ipmitool lan set 1 arp respond on
ipmitool user set password 1 admin
ipmitool lan set 1 access on
ipmitool lan set 1 user
ipmitool lan set 1 auth ADMIN PASSWORD
ipmitool user enable 1
To access it now remotely:
ipmitool -H 10.10.0.101 -P admin chassis status
Some more stuff I had to do on another system that was a Penguin machine:
ipmitool user list 2
ipmitool user priv 2 4 2 (sets userid 2 to permission4 (which is administrator) on channel 2)
QLogic debuging
Nov 24th
serial port:
screen /tty/usb.serial 57600
Or, to spell it out:
baud rate: 57600
8 bit
no parity
stop bits = 1
no flow control
Fedora 11 Kickstart and SATA drives
Nov 5th
My Fedora 11 Kickstart installs fail if I have the drive line look like this:
part / –size 1 –grow –fstype ext3
I was able to make it work by giving it a real size:
part / –size 10240 –fstype ext3
That seemed to make it work. I also noted that this bug has already been reported.
SVN with xCAT
Nov 4th
This is how I check out the latest xCAT SVN to make code updates. I just added Fedora 11 today.
svn co http://xcat.svn.sourceforge.net/svnroot/xcat/xcat-core/trunk/
svn co http://xcat.svn.sourceforge.net/svnroot/xcat/xcat-dep
svn ci anaconda.pm -m “added Fedora 11 x86_64 support”
Using IBM Bootable Media Creator with xCAT
Nov 3rd
With a few hacks I was able to get somewhere with this… I wasn’t able to update everything nor restore the bios.
1. I downloaded the tool from IBM site: http://www-947.ibm.com/systems/support/supportsite.wss/docdisplay?lndocid=MIGR-5079820&brandind=5000008
2. I put this in the /tftpboot/bomc directory.
3. Copied /tftpboot/bomc/[tcrootfs|tc.zip] into /tftpboot
4. Edited /opt/xcat/lib/perl/xCAT_plugin/dhcpd.pm line 282 made the filename say “bomc/pxelinux.0″ instead of “pxelinux.0″
5. Edited /tftpboot/bomc/pxelinux.cfg/default:
- added: console=tty0 console=ttyS1,19200
- changed tftp server: 172.20.0.1 instead of the 9.x.x.x
That got me booted up, but didn’t allow me to update the firmware… lame.
jquery ui tabs and the back button
Oct 30th
For the site I help out with, xcat.sf.net, I used jquery-ui tabs. The only issue I had was that I couldn’t do back button, and I started getting a few complaints from it…
So, after about 20 minutes of investigation, here is the code I used to make it work. It works pretty well for cases when you navigate externally, but doesn’t work when you press the back button while on the page, expecting to go to the previous tab. The jQuery team says that they’re working on this but its not in there today. Also, my approach refreshes the entire page with each click, that’s how I get the right page. So it may not be optimal for many places. Anyway, here is what I do:
<script type=”text/javascript”>
$(document).ready(function(){
var start = window.location.href.lastIndexOf(“#”);
var tabIndex = 0;
if (start !== -1){
tabIndex = window.location.href.substring(start+1, start+2);
}
$(‘#tabs’).tabs({
selected: tabIndex,
spinner: ‘Retrieving data…’,
select: function(event,ui){
window.location = “#” + ui.index;
window.location.reload();
},
});
});
</script>
Then, down below in the HTML, I have:
<div id=’tabs’>
<ul>
<li><a href=”about.html”>About</a></li>
<li><a href=”download.html”>Download</a></li>
<li><a href=”start.html”>Getting Started</a></li>
<li><a href=”docs.html”>Documentation</a></li>
<li><a href=”support.html”>Support</a></li>
<li><a href=”test.html”>Testimonials</a></li>
</ul>
</div>
That makes it so that the back button is supported. It changes the URL a bit, by giving it a nasty #<tabindex> but it does the trick for me.
IPMI settings.
Oct 29th
Some times configuring the BMC’s or IMM’s with xCAT is not possible. So you may want to do settings without xCAT.
Here are some tricks with our handy tool: ipmitool
See current IP settings:
ipmitool lan print 1
This is great for looking at IP addresses and stuff.
ipmitool -I lan -H 172.29.101.2 -P PASSW0RD -U USERID power status
KVM with RHEL5.4
Oct 29th
Installing KVM with RHEL5.4 is pretty easy. Here’s how I got my virtual machine up:
1. Install Packages
yum -y install kvm python-virtinst libvirt libvirt-python virt-manager virt-viewer
2. Start libvirt
chkconfig –level 345 libvirtd on
service libvirtd start
3. Configure Bridge Network
In my set up I want my virtual machines to be able to access the network. I also want machines on the network to access my virtual machine. In my setup, eth1 is on the internal network to my cluster. What I will do is create a bridge with eth1. I also need to make a few aliases to handle my different networks.
/etc/sysconfig/network-scripts/ifcfg-eth1:
# Intel Corporation 80003ES2LAN Gigabit Ethernet Controller (Copper)
DEVICE=eth1
HWADDR=00:15:17:85:A8:CD
ONBOOT=yes
BRIDGE=br0
/etc/sysconfig/network-scripts/ifcfg-br0
DEVICE=br0
BOOTPROTO=static
ONBOOT=yes
TYPE=Bridge
IPADDR=172.20.0.1
NETMASK=255.255.0.0
/etc/sysconfig/network-scripts/ifcfg-br0:1
DEVICE=br0:1
IPADDR=172.29.0.1
NETMASK=255.255.0.0
ONBOOT=yes
/etc/sysconfig/network-scripts/ifcfg-br0:2
DEVICE=br0:2
IPADDR=172.30.0.1
NETMASK=255.255.0.0
ONBOOT=yes
4. Create Virtual Machine
virt-install –name xcatmgr –ram 1024 –connect qemu:///system –disk path=/install/libvirt/images/xcat.img,size=10 –vnc –cdrom=/install/isos/RHEL5.4-Server-20090819.0-x86_64-DVD.iso -b br0 –os-variant=rhel5
4.2 Copying Virtual Appliance
I created an xCAT appliance that I wanted to run:
modprobe kvm
modprobe kvm.intel
service libvirshd restart
Then import my machine:
define /install/xcatapp/xcatmgr.xml
(or just copy it to /etc/libvirtd/qemu and restart libvirshd)
From there it just booted up, launched virt-viewer and away it went. I’m doing this on 1TB SATA disks, and the performance is just horrible.
Even though I did this all with RHEL4, this post was very helpful as well.
Now, I also want my nodes to bridge directly to the physical network.
service libvirtd stop
ip link set virbr0 down
brctl delbr virbr0
create br0 with /etc/sysconfig/network-scripts
bind it to eth1
then make new net:
brctl setfd vlan1 0
ip addr add dev vlan1 172.20.1.19/16
brctl addif vlan1 eth0
ip addr del dev eth0 172.20.1.19/16
ip link set vlan1 up
virt-viewer xcatmgr
Other good notes:
http://wiki.centos.org/HowTos/KVM#head-c02a0b33e7949b0bc3b151ac6e0bdfb91b6bbd1c