Converting an ext3 filesystem to ext4 on CentOS 5.6

If you recently installed CentOS 5.6 or updated to CentOS 5.6 you can now utilize the ext4 filesystem.

To do the conversion or create a new ext4 filesystem you need the e4fsprogs tool kit from yum.

#yum -y install e4fsprogs

I did this conversion on a test system with a non-root filesystem to avoid any possible problems. I also backed up the filesystem just in case something went terribly wrong.

First you need to un-mount the filesystem, as it cannot be in use.

#cd /; umount /dev/VolGroup00/LogVol00

Now you can run the tune4fs command to convert the filesystem to ext4.
#tune4fs -O extents,uninit_bg,dir_index /dev/VolGroup00/LogVol00

Now that the filesystem is ext4 it is no longer able to be mounted as ext3, so change its entry in the fstab.

#vi /etc/fstab

Now because we used the option uninit_bg we need to run fsck on the new ext4 filesystem. This is a good idea to do anyways, but is a requirement here.

#e4fsck -fDC0 /dev/VolGroup00/LogVol00

e4fsck will complain about "One or more block group descriptor checksums are invalid", this is totally normal.

Before doing these steps on the root filesystem (/) I would recommend you read over the Ext4 Howto on kernel.org. I re-wrote the steps for more accurate information pertaining specifically to CentOS 5.6 and my system.

As always:

THE INFORMATION IS DISTRIBUTED IN THE HOPE THAT IT WILL BE USEFUL, BUT WITHOUT ANY WARRANTY. IT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE INFORMATION PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.

IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW THE AUTHOR WILL BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE INFORMATION TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Update to Promiscous Mode on vSphere 4

A few months back I wrote about Enabling Promiscuous Mode on vSphere 4. Well, I have learned some more since then. There isn't much information out there about recording a VoIP stream in a virtual machine environment. I hope to change that today.

First thing is first. If you are using VLANS on your physical network set your Virtual Machine Port Group to be a trunk. Also set the port on the physical switch to be a trunk instead of an access port.

Second thing, if you are using the E1000 NIC driver in your guest OS, turn off VLAN support. The screen shot is from Windows 2008 R2 64-bit, but it is similar in every other Windows OS.







Performance Best Practices for vSphere 4

Enabling Promiscuous Mode on vSphere 4

We are in desperate need of a testing server. The testing server needs to be able to listen to the network, through port mirroring, to record VoIP calls. In the physical world this is no problem, but we don't have the budget for another physical box, especially for something as non-critical as testing a program upgrade (even though the program it self is very critical to our business).

A year ago the software vendor said that they don't support virtual environments and their product doesn't work in a virtual environment. During discussions this year however they told the truth that they simply don't know if it will work, and thus "don't support" it. I am calling it lazy. It doesn't take much these days to get a VM host going and configured for testing things out.

Well we have done the leg work for them. I have confirmed that vSphere does allow for promiscuous mode, even from the physical network. To be honest I was a little shocked to see this worked.

Here are the steps to enable promiscuous mode on your vSphere host and the guest VM.

1. Enable promiscuous mode on the virtual adapter.
   
2. Enable promiscuous mode on the vSwitch.
3. Enable promiscuous mode for the guest.
4. Enable port mirroring on the physical switch (not covered here).
5. Test capturing network data.

Now that we have the general synopsis of the procedure we can begin.

We enable promiscuous mode on the virtual adapter by logging into the vSphere Client, going to the VM Host, clicking on the "Configuration" tab, then on "Networking" in the "Hardware" section on the left. Now click on "Properties..." for the Virtual Switch.


Now enabling promiscuous mode on the vSwitch is pretty simple. Click on the "vSwitch" on the "Ports" tab, as shown below and then click on the "Edit..." button.



Now that we are editing the vSwitch properties click on the "Security" tab. Change the option for Promiscuous mode to "Accept" if it isn't already and hit "OK".


To enable promiscuous mode for the guest we need to break down to the command line. I used putty to SSH into my vSphere host, which I had previously setup. Now you need to edit the .vmx file of the guest that will be listening to the network.

# vi /vmfs/volumes/datastore1/testServer/testServer.vmx

I did a search for ethernet, so the promiscuous mode configuration would be with the rest of the ethernet config. Add the following line to the configuration file:

ethernet0.noPromisc = "FALSE"



And save when you are done. I rebooted my testServer just as a precaution, but I'm not certain it is required.

At this point everything is configured on the VM side of things. Make sure you have port mirroring enabled on your physical switch and give it a test. In my environment I commonly use Wireshark. I did my testing by pinging a server on a mirrored port.

Using PSExec to Defragment your PCs

Way back in 2000 / 2001 I was an IT intern. My job was to do the really manual processes for a small department with in a much larger company. One of those was monthly defrags of all PCs and Laptops (if they where available). I had to go to each and every PC, log on, and start the defrag process manually. I believe this to be my first moment of "I should automate this". Problem was I didn't know how or really where to start. I knew script was the answer, but just couldn't get things to work. I believe I eventually created a scheduled task on each machine (by hand) to do this for us.

Fast forward 10 years and I am still in need of the same thing, as we don't have Vista or Windows 7 deployed. But now I have a much better understanding of what needs to happen and even better I know where to start!

PSExec, which is part of the System Internals PsTools suite is my answer these days. A simple script, run from my PC (still manually for the time being) is able to handle defragmenting all of our PCs.

psexec @C:\Updates\Comps\AppPCs.txt -n 10 -c -f -d JkDefragCMD.exe

I use the files with PC names to speed up re-deployment of everything. The -n 10 flag tells psexec t0 wait 10 seconds before it times out the PC, instead of 60 (I believe this is the default). -c copies the file (jkdefragcmd.exe) to the remote system. The flag -f forces copying, even if the file exists. The flag -d doesn't wait for the process to terminate, this is as asynchronously as I can do.

My next step is to hook this up to task scheduler and have it run the first Sunday or something.

Documentation, not always the How To Dos

Documentation for me is often just How to do something, but I have been forgetting the WHY part of the equation. For instance, my documentation says to do our weekly maintenance window after 21:30 and to only reboot one particular server after 21:45, but I didn't say why and had forgotten myself. So I started doing the maintenance earlier and at one point rebooted the one particular server at 21:30. This caused the last of our production cron jobs to not run, and thus a customer didn't get their batch for the day.

Now this could have been avoided a few different ways. Since that first time was a mistake any thing that happens after it is a failure on my part, and thus can't happen again.

1. I could have followed our procedure to the T.
2. Read my e-mail to see that the cron job hadn't run yet.
3. Run the cron job by hand after the server restart.
4. Did all of the precursor work and waited for the cron job to run.
5. Did all of the precursor work and run the cron job by hand.

I have chosen to go with option 1, follow our procedure to the T. We started doing the weekly maintenance for a few reasons and it was to be done after 9:30, with that one particular server last to ensure this didn't happen. Now once again I have my WHY, and I have it written down so if I question it again I know why.


This is just one instance where the WHY is critical, but there are others. So please when you are writing documentation, and you should be please include the WHY you do something the way you do it. It also helps train the new guy, or your replacement.

Enabling Ping responses in Windows 7

Out of the box if you Ping a Windows 7 host you will recieve "Request Timed Out". This is because the Windows 7 firewall is blocking ICMP echo requests. If this is causing you problems and you need to open up access to ping requests it is fairly simple.

1. Go to the "Windows Firewall" in the Control Panel.
2. On the left hand side click "Advanced Settings"
3. Click on "Inbound Rules"
4. Right click on "Inbound Rules" and choose "New Rule"
5. Select "Custom (Custom Rule)" and press "Next"
6. Select the (default) "All Programs" and press "Next"
7. Change the Protocol type from "Any" to "ICMPv4"
8. Unless you want to restrict the ping response choose the option "Any IP address". (This is for your adapter.)
9. Unless you want to restrict which hosts can ping you choose "Any IP address".
10. On the Action screen ensure it is an "Allowed" connection.
11. Leave all three check boxes check on the Profile screen.
12. On the final screen, Name, give it a meaningful name such as "Echo Ping Request"
13. Finally click finish.
    

How to Reset the Password(s) on a Linksys SRW248G4 Switch

1. Connect to the Serial port on the back of the switch with a serial cable. (Putty on Windows works well)
2. The defaults for connecting to the serial port are 38400 baud, 8 data bits, no parity, 1 stop bit, no flow control.
3. Confirm that you have an active serial connection by pressing enter a couple of times. You should receive a login screen
4. Once you have successfully connected to the serial port restart the switch by unplugging the power. (Either from back or from the power strip, my preferred method is the back)
5. The switch will start its POST process.
6. Look for the line “Autoboot in 2 seconds - press RETURN or Esc. to abort and enter prom.” (Please note: Do not hold down Esc. or Enter. Only press it once.)
7. You will know that you interrupted the boot sequence when you are prompted with a startup menu. Select option 3 “Password Recovery Procedure”.
8. The screen will display “The current password will be ignored!”. Press Enter to reboot the switch.
9. Once the switch has restarted login with the default admin user and no password. This will work on either the terminal or the web interface. Proceed to create / modify users and passwords. This does not reset the rest of the configuration, just user accounts.
10. Once you have your new credentials reboot the switch again. Document the credentials.
11. Verify that your new credentials work by logging into either the web interface for the command line interface.
12. Disconnect the cable and store it safely.

This is the second time I have required these instructions, so I am posting them here for safe keeping. The last time I did this I removed the default account and setup individual accounts, and then promptly forgot everything about it. This is why documentation is so critical.

How to Enable Windows 7 GodMode

God mode in Windows 7 is similar to TweakUI in Windows XP, but is really simple to setup and requires no installation.

1. Create a new folder.
2. GodMode windowRename the folder to

GodMode.{ED7BA470-8E54-465E-825C-99712043E01C}

(note that you can change the “GodMode” text, but the following period and code number are required).

I create this folder in my home directory.

Backup Exec System Recovery Review

I asked a question over on Serverfault about backup software that supported Windows and Linux and did Bare-Metal Recovery and the response I got was to try Backup Exec System Recovery, which so far has been pretty good.

The bad stuff:

1) Restore CD creation on CentOS is impossible from what I have tried and can tell. They really do mean Redhat Enterprise Linux only.
2) Bare-metal recovery with MBR restore doesn't work. Which means I have to install grub after a restore.
3) LVM support is once again not there or well hidden.
4) Centralized management. It is a separate 1GB download as an Addon. I did the download and install, but now I can't find the Management console. Setting up each of our servers isn't too big of a problem, but I really wanted one window to view for backups.
5) There is no built in scheduler with Linux.
6) There is no incremental backup in Linux.

The good stuff:
1) It works on CentOS with a little bit of hacker-y to install (more on that in another post).
2) It is easy to install and configure on Windows, even though it requires a reboot. (BOO!)
3) It is pretty quick to get a backup started.
4) The backup procedure is different on Windows than on Linux. Windows is GUI based 100%. (I haven't looked to see if I can configure a backup from the command line.) While Linux is 100% CLI.
5) The Windows based restore CD is a pretty useful tool with out limits (unlike the Acronis Boot Disk).
6)Restores are quick and pretty simple to do.

All in all after trying three different products and looking at half a dozen I think this may be the winner.

CHECK_NRPE: Error - Could not complete SSL handshake

Recently I have been auditing our servers versus what we check in nagios versus what we need to do when a system is rebooted during routine maintenance. I found that two of our servers have been left out of nagios monitoring even though they are on my maintenance checklist, and they have software that I still start by hand (I KNOW IT'S NOT A BEST PRACTICE!).

Both servers had NRPE installed and configured (mostly), one of them even had a configuration file on the nagios server but it wasn't enabled (hostname.disabled instead of hostname.cfg). The other server needed a configuration file, but even then it wasn't working, so here are my troubleshooting steps:

1) Check to see that NRPE is compiled and installed. [It was]
2) Check that NRPE was listening (netstat -an | grep 5666) [It was]
3) Check that NRPE was listed in /etc/services [It wasn't]
4) Check the NRPE config file (/etc/xinetd.d/nrpe) for "only_from = 127.0.0.1 192.168.100.31" [It wasn't]

So I added the service definition to /etc/services and the nagios server IP to the only from line and restarted xinetd (service xinetd restart) and I was finally able to connect from my nagios server.

All of this is on CentOS 5.5 for both the server and the client.

Acronis Backup and Recovery Advanced Server 10

Getting up and running with Acronis Backup and Recovery Advanced Server 10 is pretty quick, especially for the demo. It took under an hour to install, configure and start backing up a Windows 2008 Standard server. Getting the Backup agent installed on our Linux servers took a little bit more work because of the SnapAPI kernel modules. After a little bit of digging around in the Acronis knowledge base I was able to resolve all of the issues I had getting the Acronis Backup Agent installed on our CentOS 5.4 servers. If we go with Acronis Backup and Recovery Advanced Server 10 I will need to add some additional lines to our Linux Post-Install script to add in the kernel-devel package and the additional RPMs that the agent needs. (DKMS and SnapAPI) both of which are already on our Storage drive.

With our second trial run here we where able to resolve most of the outstanding issues we had from the first trial. Namely excessive recovery times and recovering to dis-similar hardware. I have yet to truly attempt a Linux restore to dis-similar hardware, but I have the base system recovered and the instructions, so I will be attempting one later this week (with results to follow). Restoring our Windows machines to either dis-similar hardware or a VM is pretty straight forward with the Universal Restore CD. The main problem holding us back in those scenarios is not having the drivers readily available for post install. This can be remedied by always going to VM and installing the VM tools as they also contain the drivers for the system.

Each backup policy allows us to modify settings for the backups such as Encryption, Compression, and throttling resources. The encryption can have a separate encryption key and varying levels of encryption from none to AES-128, AES-192, and finally AES-256. The automated backups can do a simple backup plan with full and incremental that are stored as a file on the storage system. The policy also allows us to run custom commands before and after the backup so can could do a virus scan, or shred temp files or what ever we desired.

One of the strangest problems I came across was while doing the restores to VM, if started from the VMs Console after picking the Disk to recover to the process could take hours to complete. Whereas if I did the same restore on to physical hardware the process would take seconds. The work around is to boot the VM from the Universal Restore CD and use the Management Console to connect to the VM. Once connected to the VM from the Management console I was able to start the restore in a matter of minutes, more akin to a physical machine.

Something to remember when setting up the backup policy for Linux machines that utilize LVM is to back up the disk as a whole. You don't want to backup the LVM by its self because after the restore it will fail to boot.

All in all I feel that Acronis would make a wonderful addition to our eco-system. It fulfills all of the requirements of our backup scheme, except the web based access, which is not uncommon for Windows based software.

NDO2DB daemon startup script

I would like to thank Chris over at http://sysengineers.wordpress.com for the excellent post on how to daemonize NDO2DB. His post NDO2DB startup script for RH (EL) / OEL does an excellent job getting everything working. I had to make one small change to the script as I kept my ndo2db executable as ndo2db-3x instead of just ndo2db.

For me this fixes a problem where I have to remember to manually start the ndo2db service after a server restart, which hasn't been happening the past few times. The second way I am going to fix this problem is by creating a checklist of things that need to happen during a server restart / boot up sequence for each PC. This will also reduce the effort needed for when I move the rack later this year. The third way I will be checking this is by adding a check to Nagios to make sure this is running at all times.

This also illustrates my general way of checking / double checking things. One go off of a checklist. The double check is to verify that Nagios is seeing the same thing.

Installing Acronis Backup and Recovery 10 Linux Agent on CentOS 5.4

I know it has been a while since I have posted anything. I have been busy testing out backup and recovery software. The latest one is Acronis Backup and Recovery 10 Advanced Server. The management server is very easy to setup and configure. I had some issues however while installing the Agent for Linux. But with a little research I was able to find what I needed and get everything installed.

Step number one is to make sure you have the Kernel Development package for your kernel. The easiest way to find out which version you need is to run the command "uname -r". On one server I needed the "kernel-devel" package and on another I needed the "kernel-PAE-devel" package. So if you don't have them already install the correct one for your server.

After this is installed you can install another prerequisite package Dynamic Kernel Module Support Framework or DKMS for short. I got mine off of DAG's repo at http://dag.wieers.com/rpm/packages/dkms/. Once this is downloaded install the package.

The last prerequisite is an updated SnapAPI module. Thanks to the Acronis KB I was able to find the updated package at http://kb.acronis.com/sites/default/files/content/2009/10/4371/snapapi26_modules-0.7.47-1.noarch.rpm.

Now that we have all of the packages installed we can install the Agent for Linux. The install is rather painless, except the License Key that you have to type in, by hand, every time for the trial edition. Maybe its only a pain point for me because I had to do it so many times while trying to get it installed.

TL;DR

#uname -r
#yum install kernel-devel
#wget http://dag.wieers.com/rpm/packages/dkms/dkms-2.0.17.6-1.rh9.rf.noarch.rpm
#rpm -Uhv http://dag.wieers.com/rpm/packages/dkms/dkms-2.0.17.6-1.rh9.rf.noarch.rpm
#wget http://kb.acronis.com/sites/default/files/content/2009/10/4371/snapapi26_modules-0.7.47-1.noarch.rpm
#rpm -Uhv snapapi26_modules-0.7.47-1.noarch.rpm
#./AcronisAgentLinux.i686

Mounting a LVM volume in Ubuntu (Live CD)

A while back my testing server crashed. This was no surprise to anyone as it was just a (Very) old workstation. However it was running my nagios install in a production setting. I had been meaning to move it to a proper server, but just hadn't gotten around to it. To make matters worse, I didn't back any of it up. So thankfully it was only the motherboard that failed and not the HDD.

I mounted the HDD in another PC I had sitting around and booted it using Ubuntu Live CD.

First, boot Ubuntu.

Second, install the needed tools:

$ sudo apt-get install lvm2

Third, load the modules to do our task:

$ sudo modprobe dm-mod

Fourth, scan the system for LVM volumes. Look for the volumes you want to mount. Typically this will be VolGroup00:

$ sudo vgscan

Fifth, we need to activate the volume(s):

$ sudo vgtchange -ay VolGroup00

Sixth, Look for the logical volume containing the root file system. Typically this will be LogVol00:

$ sudo lvs

Seventh, create the directory to mount the drive:

$ sudo mkdir /mnt/restore

Eighth, Mount the volume to the directory you just created.

$ sudo mount /dev/VolGroup00/LogVol00 /mnt/restore -o ro,user

Ninth, Copy your files off of the drive.

$ cp /mnt/restore/some/dir/and/path /some/dir/and/path

Tenth, Setup what ever backup means you have on the new server!

All in all this wasn't a terrible thing, it could have been much worse. I have since moved the nagios setup to a virtual machine and am backing it up nightly.

Upgrading the Firmware on a 3com 4500 switch

Again, this is mostly for my own notes, but someone else may find it useful. Last year we purchased a 3c0m POE switch for our new VoIP phone system. This year I needed to update the firmware on it, but with only 8MB of flash drive I ran into a few problems.

First off, BACK UP EVERYTHING!

I used the TFTP method to transfer files to and from the switch. I used Solarwinds TFTP Server on my PC.

File name Prefix / Suffix

s3n / .app = 4500 application software.

s30 / .btm = 4500 boot ROM software

s3p / .web = 4500 web file (HTTP management interface)

3comOScfg.def / .def = 4500 config file

So first things first backing up via TFTP:

<4500>tftp [IP OF TFTP SERVER] put flash:/s3004_01.btm

<4500>tftp [IP OF TFTP SERVER] put flash:/s3p04_03.web

<4500>tftp [IP OF TFTP SERVER] put flash:/s3n03_03_02s56p05.app

<4500>tftp [IP OF TFTP SERVER] put flash:/3comOScfg.def

Please change these files as you see fit. Do a dir on the root directory to get the listing for your particular switch.

Now that we have that backed up we need to clean up the flash:/ drive to make room for the updates.

<4500>delete s3004_01.btm

<4500>delete s3p04_03.web

<4500>detete s3n03_03_02s56p05.app

Now we also have to empty the recycle-bin. This is where I got stuck as I didn't know a CLI could have a recycle-bin, or have never seen it done before.

<4500>reset recycle-bin

See as we now have the free space we need to get the new files we can pull them down from the TFTP server.

<4500>tftp [IP OF TFTP SERVER] get s3p02_01.web

<4500>tftp [IP OF TFTP SERVER] get s3o01_01.btm

<4500>tftp [IP OF TFTP SERVER] get s3n03_02_00s56.app

Again, you will need to use the files that are the current firmware update.

One of the last steps is to tell the switch what files to use on next boot.

<4500>boot boot-loader flash:/s3n03_02_00s56.app

<4500>boot bootrom flash:/s3o02_01.btm

Finally we will save the configuration and reboot the switch.

<4500>save

<4500>reboot

Thats it! After the switch reboots you will be running the newest software, except for the .web file. For some reason this is left out of all of the documentation that comes with the update. I didn't write down the commands I used to get it to update, and I honestly don't think they worked.

Locking Down Mozilla Firefox

One of the main draw backs to Mozilla Firefox from most corporate IT people is its inability to be locked down, or taken care of by Group Policy. While you cannot lock down Firefox from within Group Policy alone, you can script these fixes into place to lock settings down.

To start you will need a program to byte shift a config file so that Firefox can read it. I used Byte Shifter.exe. There are also websites that do it all in a browser, but I haven't used them.

You will also need to edit the all.js file in "C:\Program Files\Mozilla Firefox\greprefs\" to include:

pref("general.config.filename", "mozilla.cfg");

I put that line at the absolute bottom, but it might not matter where it goes.

To start create an empty file. I called mine mozilla.txt since the result file will be mozilla.cfg.

The file must start with // on its own line.

Add in any settings you want to lock down. You can peruse about:config for settings, and again after you lock them down to check for the "locked" option.

Below is my mozilla.txt file, with host names changed to protect the innocent. I have included comments (They start with //) to explain things a bit further.

//

//Lock the option for startup page. 0 = "Show a blank page", 1 = "Show my home page", and 3 = " Show my windows and tabs from last time"

lockPref("browser.startup.page", 1);

//Set the home page. Use a pipe to include many home pages as tabs.

lockPref("browser.startup.homepage", "http://www.mysuperawesomecompany.com|https://mail.google.com");

//Set the browser history to something a bit longer than the default 7 days.

lockPref("browser.history_expire_days", 90);

lockPref("browser.history_expire_days.mirror", 90);

// Clean up certain things every time Firefox shuts down. This keeps things clean and running smooth for us, your results may very.

lockPref("privacy.sanitize.sanitizeOnShutdown", true);

//We do not want to clear the history on shutdown.

lockPref("privacy.clearOnShutdown.history", false);

lockPref("privacy.item.history", false);

//We can clean up the downloads history. I have seen things get really slow if this doesn't happen.

lockPref("privacy.item.downloads", true);

//Clear the cache.

lockPref("privacy.item.cache", true);

//Clean up cookies.

lockPref("privacy.item.cookies", true);

//Remove any session info.

lockPref("privacy.item.sessions", true);

//We do not want to keep passwords saved.

lockPref("privacy.item.passwords", true);

//Do not prompt to do this, just do it.

lockPref("privacy.sanitize.promptOnSanitize", false);

lockPref("signon.rememberSignons", true);

//Do not allow the "Show passwords" button.

lockPref("pref.privacy.disable_button.view_passwords", true);

//Don't use a proxy.

lockPref("network.proxy.type", 0);

//We keep one version of Firefox for a while. The newest version breaks things in our application, so we currently need to just run what we have.

lockPref("app.update.enabled", false);

//Disable extensions.

lockPref("config.lockdown.disable_extensions", true);

//Disable themes.

lockPref("config.lockdown.disable_themes", true);

//Show the downloads window when downloading a file.

lockPref("browser.download.manager.showWhenStarting", false);

//Close the downloads window when all downloads are done.

lockPref("browser.download.manager.closeWhenDone", true);

//Save files to:

lockPref("browser.download.useDownloadDir", true);

lockPref("browser.download.dir", "c:\\%homepath%\\Desktop");

lockPref("browser.download.downloadDir", "c:\\%homepath%\\Desktop");

lock{ref("browser.download.folderList", 2);

//Always ask me where to save files.

lockPref("browser.download.useDownloadDir", false);

//Always check to see if Firefox is the default browser.

lockPref("browser.shell.checkDefaultBrowser", false);

//New pages should open in a new window.

lockPref("browser.link.open_external", 2);

lockPref("browser.link.open_newwindow", 2);

//New pages should open in a new tab.

lockPref("browser.link.open_external", 1);

lockPref("browser.link.open_newwindow", 1);

//Warn me when closing multiple tabs.

lockPref("browser.tabs.warnOnClose", false);

//Warn me when openining multiple tabs might slow down Firefox.

lockPref("browser.tabs.warnOnOpen", false);

//Always show the tab bar.

lockPref("browser.tabs.autoHide", false);

//When I open a link in a new tab, switch to it immediately.

lockPref("browser.tabs.loadInBackground", false);

//Block pop-up windows.

lockPref("dom.disable_open_during_load", false);

//Load images automatically. 1 = check 2 = unchecked.

lockPref("permissions.default.image", 2);

//enable JavaScript.

lockPref("javascript.enabled", true);

//Some of the advanced JavaScript options.

//Disable the Advanced Button.

lockPref("pref.advanced.javascript.disable_button.advanced", true);

//Move or resize existing windows.

lockPref("dom.disable_window_move_resize", true);

//Raise or lower windows.

lockPref("dom.disable_windows_flip", false);

//Disable or replace context menus.

lockPref("dom.event.contextmenu.enabled", false);

//Hide the status bar.

lockPref("dom.disable_window_open_feature.status", false);

//Enable Java.

lockPref("security.enable_java", false);

Of course there are others and many more settings, but that covers a good number of them. Also I have only tested this on our version of Firefox, which is 2.0.0.20.

Nagios: check_http, using the --invert-regex option

Some times you want to check that something is running or working correctly and you work out tests for that. Other times you want to know when something is broken and throwing error messages. This is about the latter, a proper HTTP 200 code is great and all, but what if the page is just showing "Too Many Connections" instead of your home page? My old check_http command for this server used to look like, well, check_http. I didn't check anything about it specifically, just that it was returning a 200 code.

Today however I knew I needed something more in depth. Our database server lost its local network connection, but still was available over the public IP, which is what I test against. Once we re-directed the SQL requests to the public IP address of the server everything started working again, until we ran across "Too Many Connections". The database server kept all of the "local" connections open and thus we ate up the rest.

So, how to test for this scenario? After reading through the man pages of check_http I saw this little gem "--invert-regex Return CRITICAL if found, OK if not". This I knew was exactly what I was looking for! If it sees our error codes it will go Critical! Now to put this gem into practice. Here is where the man pages fall short. There is no explanation on HOW to use this, just that it exists. I tried the obvious to me "check_http -H hostname.com -w 3 -c 5 --invert-regex 'Some string'", but that didn't work. OK, lets try "check_http -H hostname.com -w 3 -c 5 --invert-regex='Some string'" nope that errored out with " option `--invert-regex' doesn't allow an argument".

Third times the charm right?

"check_http -H hostname.com -w 3 -c 5 -r 'Some string' --invert-regex '"

# HTTP OK HTTP/1.1 200 OK - 0.355 second response time |time=0.354966s;3.000000;5.000000;0.000000 size=12975B;;;0

Yes, as it turns out third time is the charm. So that got me thinking some more. How can I ensure that the page is rendering correctly, and if it isn't fail but in a specific way?

"check_http -H hostname.com -w 3 -c 5 -r 'Some string I want in my page' -r 'Some string I don't want to see' --invert-regex '"

You can add more than one -r to the check_http command and it will require all of them to be present for the test to pass, and if one of them fails then it will go critical! Perfect!

If you have any more insight into using the check_http command in Nagios I want to hear about it. We are always running into new failure scenarios that we didn't anticipate and I want to know about them before one of my users tells me about it.

Bash: Finding files between two dates in the current directory

Today my boss asked me for a bash command (or script) to find some files between two dates.

Thanks to Jadu Saikia over at Unstableme his post UNIX BASH scripting: Find Files between two dates, I had a starting point.

This will find all files between the two dates (20071019 & 20071121) in this case.

find . -type f -exec ls -l --time-style=full-iso {} \; | awk '{print $6,$NF}' | awk '{gsub(/-/,"",$1);print}' | awk '$1>= 20071019 && $1<= 20071121 {print $2}'

Now, if you want just PGP files you would do:

find *.pgp -type f -exec ls -l --time-style=full-iso {} \; | awk '{print $6,$NF}' | awk '{gsub(/-/,"",$1);print}' | awk '$1>= 20071019 && $1<= 20071121 {print $2}'

The second request that my boss was looking for with this is the file size, something that was being left out by awk. So we can fix that by updating the command to:

find *.pgp -type f -exec ls -lh --time-style=full-iso {} \; | awk '{print $6,$NF,$5}' | awk '{gsub(/-/,"",$1);print}' | awk '$1>= 20090624 && $1<= 20090901 {print $2,$3}'

We added in a $5 to the first awk command, and the final one had $3 added to it. Also I like human readable file sizes so I added -h to the ls command.

Creating a RAID 5 Array in software on CentOS 5.3

In order to create a RAID 5 array entirely in software on Linux you need to do a few things.

First I used three Identical drives, same speed, size, make, and model. This may not be a requirement, but

it will defiantly help the process. For RAID 5 you will need at least three partitions of the same size.

I picked up a four disk internal hot-swap enclosure from Addonics (http://www.addonics.com/products/raid_system/ae4rcs35nsa.asp) and

a hot-swap capable raid card. Once I got everything physically installed in the server case I booted into CentOS 5.3

and got on the command line. The first thing you need to do is create partitions on the blank drives and set them to be

Linux raid autodetect (Hex value of fd). To do this run fdisk /dev/sdX where X is the drive(s) that you want to partition for the RAID array.

For me this was SDC, SDD, SDE but you results will vary.

fdisk /dev/sdc

n [ENTER]

e [ENTER]

4 [ENTER]

[ENTER]

[ENTER]

n [ENTER]

l [ENTER]

t [ENTER]

5 [ENTER]

fd [ENTER]

w [ENTER]

You will want to create an Extended partition so choose n for New partition. If you get stuck you can hit m to get the help menu.

After you hit n type e for Extended. Then you will have to enter a partition number, I choose 4 for all of my drives.

After that you n again and then choose l for logical partition type. Again it will ask for a partition number 1-4 are for primary and thus not an option,

so I choose 5 for all of my drives.

Once the logical partitions are created hit t to change the partition type. If you are unsure what to use hit l, but in this case we already know that we want to

use type fd for the Linux RAID auto.

Once the type has been changed you can type w to write this info to the drive and start on the next one.

fdisk /dev/sdd

n [ENTER]

e [ENTER]

4 [ENTER]

[ENTER]

[ENTER]

n [ENTER]

l [ENTER]

t [ENTER]

5 [ENTER]

fd [ENTER]

w [ENTER]

fdisk /dev/sde

n [ENTER]

e [ENTER]

4 [ENTER]

[ENTER]

[ENTER]

n [ENTER]

l [ENTER]

t [ENTER]

5 [ENTER]

fd [ENTER]

w [ENTER]

Now that we have three (the minimum) partitions for our RAID device we have to create it in CentOS. Here we will use a tool called mdadm to create the

actual RAID device in Linux.

/sbin/mdadm --create --verbose /dev/md0 --level=5 --raid-devices=3 /dev/sdc5 /dev/sdd5 /dev/sde5 [ENTER]

Once this returns back, and it should be pretty quick, you will have a RAID 5 device. To check the status of it run:

/sbin/mdadm --detail /dev/md0

/dev/md0:

Version : 00.90.03

Creation Time : Wed Jul 8 09:14:19 2009

Raid Level : raid5

Array Size : 2930271744 (2794.52 GiB 3000.60 GB)

Used Dev Size : 1465135872 (1397.26 GiB 1500.30 GB)

Raid Devices : 3

Total Devices : 3

Preferred Minor : 0

Persistence : Superblock is persistent

Update Time : Fri Jul 10 11:20:02 2009

State : clean

Active Devices : 3

Working Devices : 3

Failed Devices : 0

Spare Devices : 0

Layout : left-symmetric

Chunk Size : 64K

UUID : 53f6f95a:9e33f5ba:7ac8ef3e:0a40921a

Events : 0.2

Number Major Minor RaidDevice State

0 8 21 0 active sync /dev/sdc5

1 8 37 1 active sync /dev/sdd5

2 8 53 2 active sync /dev/sde5

Now for me after I first created the RAID device the state was listed as clean, degraded, rebuilding. There was a place for the percentage rebuilt and

/dev/sde5 was listed as spare. It took the better part of a day for the rebuild to finish, given that it was a three terabyte device I am not surprised by that.

Once the rebuild was done I had to create a Physical volume for LVM to be able to manage the RAID device.

pvcreate /dev/md0

After that I was able to use system-config-lvm to create the storage volume and format the drive.

Copy Directory Structure Only

This is a simple one liner for copying a Directory structure, and not the contents.

find * -type d -exec mkdir /new_directory/\{\} \;

Now there are a few caveots to this of course, but they are simple.

First the /new_directory/ has to exist.

Second, you have to run the command from within the directory that you want to copy the structure from.

For example, if I need to copy the structure of /Storage to /newStorage I would:

mkdir /newStorage

cd /Storage

find * -type d -exec mkdir /newStorage/\{\} \;