01 Oct 07 Welcome to hpux.ws
This site was created for two reasons. It was meant to document procedures, scripts and processes that I felt were important to myself and the greater HP-UX community.
This project is owned by ISN Corporation, a Subchapter S Corporation based in Chicago, Illinois, USA.
If you want to contribute to the site, let me know. hpuxadmin in gtalk hpuxconsulting in yahoo messenger
Regards,
Steven “Shmuel” Protter: Rosh Tzurim, Israel
Tags: blog, forums.itrc.hp.com, hpux, itrc
17 Jun 20 HP-UX Serviceguard Missing node
If you have a hardware fault or other calamity in a HP-UX serviceguard cluster you lose the ability to make incremental changes to the cluster until that node comes back.
If you need to make a change to a cluster in this state and you don’t want to bring down the cluster, you have to do all your changes with one gigantic command line.
Lets say you have a 4 node cluster named cnode1,cnode2,cnode3, and cnode4.
cnode4 suffers a hardware fault and you packages fail over to cnode1-3. But your usage has grown and you have a package that is beating the hardware down and you want to move it from cnode2 to cnode1.
Well you can’t do it incrementally. You have to do it all at once. I recently ran into a situation where I had to modify 37 cluster environment files and the cluster configuration to remove a node cnode4.
That requires you to correctly type a command line that could easily be in excess of 4000 characters. Anybody who knows my typing skills knows this is beyond my abilities on my best day.
So I wrote a little assistant program.
It consists of three files two of which are scripts.
pkg-mod-list (A list of all the package configuration files, full path that need to be modified. It is your choice how to handle the editing. We used ansible last night when we did it in a DR cluster.
Contents …
/etc/cmcluster/nc-package-name/nc-package-name.env
/etc/cmcluster/sc-package-name/sc-package-name.env
Then we have helper scripts which put the command line together.
myclusterV6_prod.conf is the main cluster configuration file with the references to node cnode4 commented out.
cat missing-node-checkconf
MAIN=”cmcheckconf -C /etc/cmcluster/configs/myclusterV6_Prod.conf”
PCMD=””
cat pkg-mod-list | while read -r pfile
do
PCMD=”${PCMD} -P ${pfile}”
### echo “$PCMD”
done
MYCMD=”${MAIN} ${PCMD}”
echo $MYCMD
exec ${MYCMD}
MAIN=”cmapplyconf -C /etc/cmcluster/configs/myclusterV6_Prod.conf”
PCMD=””
cat pkg-mod-list | while read -r pfile
do
PCMD=”${PCMD} -P ${pfile}”
### echo “$PCMD”
done
MYCMD=”${MAIN} ${PCMD}”
echo $MYCMD
exec ${MYCMD}
13 Nov 19 Using File Descriptors other than stdin/stdout/stderr in Shell Scriptings
For a longish ime, I had to jump through major hoops to script around the issue of my standard input getting clobbered when inside a loop that was iterating over something that coming from stdin. I don’t have an exact example that recreates the issue but something like this would generate lots of headaches:
cat <some file> | awk '{<some fancy awk-type hingys>}' | \
while read entry; do
bla; bla; bla
some_command_here_that_would_whack_stdin
bla; bla; bla
doneSorry I cannot provide an actual snippet of code to recreate the issue but what I would find in these situations is that my loop would end after one iteration (when I knew there should have been a lot more) and I coudn’t figure out why which made me haz a sad.
Well, I found a way to steer clear of all that by assigning a <some file> to a file descriptor different from stdin. To wit:
exec 3< /path/to/file
while read entry <&3; do
bla; bla; bla
some_command_here_that_would_whack_stdin
bla; bla; bla
doneThere are no doubt other ways to solve this conundrum but this is the way I have avoided it for quite some time now. Of course, make sure you do not associate a file with file descriptors 0,1,2 (unless you are quite sure that is what you want to do!).
13 Nov 19 Convert LVM volume group from 1.0 to 2.2
# First – show proof that we are currently using VG Version 1.0
# vgdisplay /dev/vgscott01
--- Volume groups ---
VG Name /dev/vgscott01
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 3
Open LV 3
Max PV 32
Cur PV 16
Act PV 16
Max PE per PV 40960
VGDA 32
PE Size (Mbytes) 32
Total PE 131204
Alloc PE 65849
Free PE 65355
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0
VG Version 1.0 <----- Proof :-)
VG Max Size 40t
VG Max Extents 1310720
--------------------------------------
# Step #1. From experience, we *know* that each current disk in the VG will
# need to have at least one free extent at the end to accomodate the transition
# to VG Version 2.x.
#
# But, currently, all extents from all current disks are allocated.
# We can move off 'N' number of extents from the end of the current
# disk to a new disk using the "-e" option of the pvmove command.
#
# (note: we use dsf's for all commands)
--------------------------------------
#
# First we need to unmount all filesystems
#
umount /bb/9/dev/mylab
umount /bb/9/dev/venturelab
umount /bb/9/dev
--------------------------------------
#
# Ok, we need to free up at least one extent from the end of
# the original, fully-qualified disks.
#
# We use the "-e" option of the pvmove command to make this happen.
# pvmove -e 2 /dev/disk/disk247 /dev/disk/disk265
# pvmove -e 2 /dev/disk/disk248 /dev/disk/disk265
# pvmove -e 2 /dev/disk/disk251 /dev/disk/disk265
# pvmove -e 2 /dev/disk/disk249 /dev/disk/disk265
# pvmove -e 2 /dev/disk/disk260 /dev/disk/disk265
# pvmove -e 2 /dev/disk/disk259 /dev/disk/disk265
# pvmove -e 2 /dev/disk/disk258 /dev/disk/disk265
# pvmove -e 2 /dev/disk/disk261 /dev/disk/disk265
# pvmove -e 2 /dev/disk/disk262 /dev/disk/disk265
# pvmove -e 2 /dev/disk/disk263 /dev/disk/disk265
# pvmove -e 2 /dev/disk/disk264 /dev/disk/disk265
#
# At this point, a vgdisplay should show at least 2 extents for each physical volume.
# Check our work
#
# vgdisplay -v /dev/vgscott01
--------------------------------------
# vgversion -r -V 2.1 /dev/vgscott01
#
# We are good to go!
#
#
# We need to deactivate the Volume Group before we can upgrade
#
# vgchange -a n /dev/vgscott01
#
# We are good to go!
#
# vgversion -V 2.1 /dev/vgscott01
--------------------------------------
#
# Check our work
#
# vgchange -a y /dev/vgscott01
# vgdisplay -v /dev/vgscott01
--------------------------------------
#
# Now, we can extend our logical volume
#
# lvdisplay /dev/vgscott01/lvol1
# lvextend -l 98864 /dev/vgscott01/lvol1
# fsadm -F vxfs -b 3163648m /bb/9/dev
--------------------------------------
# lvdisplay /dev/vgscott01/lvol1
--------------------------------------
# vgcfgbackup /dev/vgscott01
--------------------------------------
# mount /bb/9/dev
# mount /bb/9/dev/mylab
# mount /bb/9/dev/venturelab
# bdf /bb/9/dev
10 Nov 19 How to be a yes man
Learning something new is great. Joe Geiger taught me something cool that I should have learned years ago.
Serviceguard users ever wanted to script a cluster change such as a node add.
cmapplyconf -v -P <package file>
Ends with a y/n prompt do you want to apply? Normally that requires input. Not with the yes command:
cmcheckconf -v -P <package file>
rc=$?
# Check return code if not zero stop
if [ ${rc} -ne 0 ]
then
echo “Checkconf error ${rc}”
exit ${rc}
fi
yes | cmapplyconf -v -P <package file>
# Check return code here as well
Tags: automation, HP-UX, scripting, serviceguard, yes command
03 Sep 19 Add new dump device
add device called lvdump1
[root@stlam61p]
:/home/root # lvchange -r n /dev/vg00/lvdump1
Set dump lv to contiguous. SEP
lvcreate -L <size in MB> -n lvdump1 /dev/vg00
# lvchange -C y /dev/vg00/lvdump1
Logical volume “/dev/vg00/lvdump1” has been successfully changed.
Volume Group configuration for /dev/vg00 has been saved in /etc/lvmconf/vg00.conf
# crashconf -a /dev/vg00/lvdump1
:/home/root # crashconf -v
15 Aug 19 Turning of hardware test at boot on BL class Itanium blades
The Before picture:
BOOTTEST Settings Default Variable OS is not speedy boot aware. Selftest Setting
——— ————–
early_cpu Run this test
late_cpu Run this test
platform Run this test
chipset Run this test
io_hw Run this test
mem_init Run this test
mem_test Run this test
BOOTTEST Settings Default Variable OS is not speedy boot aware. Selftest Setting
Shell> boottest
Shell> boottest mem_test off
Shell> boottest mem_init off
Shell> boottest io_hw off
Shell> boottest chipset off
Shell> boottest platform off
Shell> boottest late_cpu off
Shell> boottest early_cpu off
BOOTTEST Settings Default Variable OS is not speedy boot aware. Selftest Setting
——— ————–
early_cpu Skip this test
late_cpu Skip this test
platform Skip this test
chipset Skip this test
io_hw Skip this test
mem_init Skip this test
mem_test Skip this test
Skip this test Shell>
This cuts boot time over 90%. NOTE: YHou may need to turn this stuff back on to troubleshoot hardware issues.
15 Aug 19 APA network pairings: How to find out fast what they are
Script for detecting APA network bonded pairs. It is already built into the cinam21t drd image. It will save you 3-5 hours of guess work on future builds.
Networking was changed to protect the innocent.
Here is an example:
| [root@cinam21t]:/home/root # ./apanetwork_discover 142.18.1.26 142.18.1.96 ——————————————————— -This script figures out which NIC cards are APA paired.- -It has two inputs:……………………………….- -1- The assigned IP address of the APA Group lan90#…..- -2- The known network address of an HP-UX server on net.- -ex ./apanetwork_discover 142.18.1.26 142.18.1.96 ……- – These are cinam21t and stlam31t…………………..- – The system must be OFF network for this to work ……- – Instruction: …………………………………..- – /sbin/init.d/net stop …………………………..- – /sbin/init.d/vlan stop ………………………….- – /sbin/init.d/hplm stop ………………………….- – /sbin/init.d/hpapa stop (You may need to ctrl-break…- – netstat -rn (ifconfig lan# down then unplumb any lans.- – Wash,rinse and repeat for lan901,lan902,lan903 …….- ——————————————————— The LAN is lan0 Success lan0 as 142.18.1.26 was able to ping 142.18.1.96 The LAN is lan8 NO JOY lan8 as 142.18.1.26 was able NOT to ping 142.18.1.96 The LAN is lan16 NO JOY lan16 as 142.18.1.26 was able NOT to ping 142.18.1.96 The LAN is lan19 NO JOY lan19 as 142.18.1.26 was able NOT to ping 142.18.1.96 The LAN is lan2 NO JOY lan2 as 142.18.1.26 was able NOT to ping 142.18.1.96 The LAN is lan49 NO JOY lan49 as 142.18.1.26 was able NOT to ping 142.18.1.96 The LAN is lan52 NO JOY lan52 as 142.18.1.26 was able NOT to ping 142.18.1.96 The LAN is lan56 Success lan56 as 142.18.1.26 was able to ping 142.18.1.96 [root@cinam21t]:/home/root # |
In this case lan0 are in the bonded pair (lan900)
Take a nwmgr output before bringing network down. Run from console only
Here is the script code
/root/build # cat apanetwork_discover
!/bin/ksh
#
echo “———————————————————“
echo “-This script figures out which NIC cards are APA paired.-“
echo “-It has two inputs:……………………………….-“
echo “-1- The assigned IP address of the APA Group lan90#…..-“
echo “-2- The known network address of an HP-UX server on net.-“
echo “-ex ./apanetwork_discover 172.19.1.26 172.19.1.96 ……-“
echo “- These are stlam34t and stlam31t…………………..-“
echo “- The system must be OFF network for this to work ……-“
echo “- Instruction: …………………………………..-“
echo “- /sbin/init.d/net stop …………………………..-“
echo “- /sbin/init.d/vlan stop ………………………….-“
echo “- /sbin/init.d/hplm stop ………………………….-“
echo “- /sbin/init.d/hpapa stop (You may need to ctrl-break…-“
echo “- netstat -rn (ifconfig lan# down then unplumb any lans.-“
echo “- Wash,rinse and repeat for lan901,lan902,lan903 …….-“
echo “———————————————————“
IP2=$2
IPADDY=$1
nwmgr | awk ‘!/hp_apa/{ printf “%s %s\n”, $1,$2 }’ | awk ‘/UP/{print $1}’ | while read -r LN
do
sleep 1
echo "The LAN is ${LN}"
ifconfig ${LN} ${IPADDY} netmask 255.255.255.0 up > /dev/null
ping ${IP2} -n 1 -m 5 > /dev/null
rc=$?
if [ $rc -eq 0 ]
then
echo "Success $LN as $IPADDY was able to ping $IP2"
else
echo "NO JOY $LN as $IPADDY was able NOT to ping $IP2"
fi
ifconfig ${LN} down
ifconfig ${LN} unplumbdone
04 Oct 18 Glance/midaemons wont start
Troublehooting steps:
1- Remove the /var/opt/perf/ttd.pid and try to start glance again
#rm /var/opt/perf/ttd.pid
#glance
2- If the above fails to fix it then stop and restart Glance as follows
# mwa stop
# midaemon -smdvss 4M -kths 1000 -pids 5000 -p # ps -ef | grep midaemon Make sure the midaemon is running, # mwa start
Modify MWA_START_COMMAND variable in /etc/rc.config.d/ovpa as follows to keep the changes across system reboot.
# grep MWA_START_COMMAND /etc/rc.config.d/ovpa MWA_START_COMMAND=”/opt/perf/bin/midaemon -smdvss 4M -kths 1000 -pids 5000 -p ; /opt/perf/bin/mwa start”
Tags: HP-UX, hpux, hpux 11.31, ia64, patches
13 Aug 18 Disk is hung by drd nothing works
claimed on ioscan
fuser -c shows clean
drd is insists the disk is busy
DRD crashed causing the issue.
Don’t want to reboot that is an admission of defeat.
ERROR: Analysis of file system creation fails.
– Analysis of target fails.
– Analysis of the configuration with disk “/dev/disk/disk143” fails.
– The analysis step for creation of an inactive system image failed.
– The default DRD mount point “/var/opt/drd/mnts/sysimage_001/” cannot be used due to the following error(s):
– The mount point /var/opt/drd/mnts/sysimage_001/ is not an empty directory as required.
* Analyzing For System Image Cloning failed with 1 error.
* DRD operation failed, contents of /var/opt/drd/tmp copied to /var/opt/drd/save.
======= 08/13/18 06:39:16 EDT END Clone System Image failed with 1 error. (user=hcladmin) (jobid=ohonq001)
cd /var/opt/drd/mnts/
rm -rf *
scsimgr clear_kmstat -D /dev/rdisk/disk143
scsimgr: Cleared the Kmetric data successfully
DRD nirvana.
If this solution helped you consider making a donation to support the site:
Tags: drd, hpux 11.31, LVM, scsimgr
Recent Tweets
https://www.youtube.com/watch?v=FR7tKD-7ht8
CNW 935 O Gauge Legacy locomotive review https://youtu.be/fpmy6gNyiNk via @YouTube
Just another train video https://youtu.be/IEmWaisFSgU via @YouTube
