[Solved] “Logical volume is used by another device”

tl;dr: use dmsetup remove before trying lvremove

Note: Volume group and logical volume names have been substituted here. I’m not entirely sure it’s necessary, but better safe than sorry. If following this, please use the names of your volume group[s] and logical volume[s]

I am in the process of combining fileserver information, and so I have been touching parts of the system not usually looked at in the normal case of day-to-day operations. For some reason, on one of my logical volumes I had created a partition table and added a partition. Of course, that worked normally so there was no reason to be aware of this — clearly I had blanked the fact that I did it at all not long after doing so — until recently.

The Problem

Logical volume vg/lv-old is used by another device.

After copying the data over to a new logical volume, I wanted to remove the now-unnecessary original logical volume that contained the partition. Easy, right?


# lvremove -v /dev/vg/lv-old
    DEGRADED MODE. Incomplete RAID LVs will be processed.
    Using logical volume(s) on command line
  Logical volume vg/lv-old is used by another device.

Okay, what’s using it? cat /proc/mounts reports that it isn’t mounted. lsof and fuser return nothing. Maybe retrying the command will work*… nope.

There are a bunch of posts around this, mostly saying “make sure it is umounted first”, or “try using -f with lvremove“. And the old favourite: “a reboot fixed it”.

Find Out device-mapper’s Mapping

Well, the culprit in this case seemed to be device-mapper creating a mapping which counted as ‘in-use’. Check for the mapping via:


# dmsetup info -c | grep old
vg-lv--old       253   9 L--w    1    2      1 LVM-6O3jLvI6ZR3fg6ZpMgTlkqAudvgkfphCyPcP8AwpU2H57VjVBNmFBpL
Tis8ia0NE

Find Out Mapped Device

Then use that to find out what is holding it:


$ ls -la /sys/dev/block/253\:9/holders

drwxr-xr-x 2 root root 0 Dec 12 01:07 .
drwxr-xr-x 8 root root 0 Dec 12 01:07 ..
lrwxrwxrwx 1 root root 0 Dec 12 01:07 dm-18 -> ../../dm-18

Remove Device (via `dmsetup remove`)

Then do a dmsetup remove on that device-mapper device:


# dmsetup remove /dev/dm-18

Retry `lvremove`

And you’re good to go with lvremove:


# lvremove -v /dev/vgraid6/lv-old
    DEGRADED MODE. Incomplete RAID LVs will be processed.
    Using logical volume(s) on command line
Do you really want to remove active logical volume lv-old? [y/n]: y
    Archiving volume group "vg" metadata (seqno 35).
    Removing vg-lv--old (253:9)
    Releasing logical volume "lv-old"
    Creating volume group backup "/etc/lvm/backup/vg" (seqno 36).
  Logical volume "lv-old" successfully removed

Bish bash bosh!

Addendum

*: I’m not sure of the thought process behind “just try it again”.

I’m reminded of a short bit of Darrell Hammond’s stand up (paraphrased):

“You know that message you get when you dial the wrong number that tells you to ‘check you have the right number and dial again’? Well, women will check the number and try again. Men will try the same number, but this time we’ll push the buttons a ******** harder…”

Browsing MySQL Backups

tl;dr: Seems the quickest way of doing this was to fire up a VM, install mysql-server and mysql-client and browse that way.

I have backups of things. This is important, because as the old adage goes: running without backups is data loss waiting to happen. I’m not sure if that’s the adage, but it’s something resembling what I say to people. I’m a real hit at parties.

I wanted to check the backups of the database powering this blog, as there was a post that could swear I remembered referring to (iterating over files in bash) but couldn’t find. I had a gzipped dump of the MySQL database, and wanted to check that.

zgrep bash mysql.sql.gz | less was my first thought, but that gave me a huge amount of irrelevant stuff.

A few iterations later and I was at zgrep bash mysql.sql.gz | grep -i iterate | grep -i files | grep -v comments and none the wiser. I had hoped there was some tool to perform arbitrary queries on dump files, rather than going through a proper database server, but that’s basically sqlite and to my limited searches, didn’t seem to exist for MySQL.

What I ended up doing was firing up a VM, installing mysql-server and mysql-client and dumping the dump into that server via zcat:

zcat mysql.sql.gz | mysql -u 'root' -p database

And then querying the database: select post_title, post_date from wp_posts where post_title like '%bash%' followed by select post_content from wp_posts where post_title like '%terate%';

And the post is back!

Show Progress Bar During dd Copy

There are a number of ways of showing the progress of a dd copy. The easiest is sending the USER1 signal to the dd process, like:

dd if=FILE1 of=FILE2
pkill -USER1 dd

But that only gives a current status – eg 12345678 bytes transferred (11.77MB) … [8.56MB/s]. Not that helpful if you want an ongoing update. You can make it periodic by using the watch command:

watch -n 10 pkill -USR1 dd

Still not perfect. If you want a progress bar, ETA and so forth, you’re best off using pv, a utility that measures the speed of a file through a pipe. If you don’t know what that means, I’d recommend reading up on the UNIX philosophy and pipes, but basically it means you can tell what dd is doing. Use it like so:

pv FILE1 | dd of=FILE2

For my use, backing up a 500GB hard drive that I use with a NAS (an NSLU2 I’ve probably mentioned before), I used the following command:

sudo pv /dev/sde3 |dd of=~/tera/nslu2.img

/dev/de3 is the data partition on the drive
~/tera/nslu2.img is the image file I want written to a terabyte-sized hard drive, mounted at ~/tera/.

As a post-script, the reason I’m backing this drive up is my nslu2 is failing weirdly. It was running without a hitch for years, then it without apparent warning dismounted the drive in slot1 and reverted to running from flash. I only figured that out as I had to log in with an old password. Trying to start it up results in it beeping once every minute or so, with the ready status lamp flashing orange. It seems to go through a loop. I thought it was a temporary glitch, as when I plugged the drive into my desktop to check the thing was intact the partitions showed up, then when I plugged it back into the nslu2 and turned it on it worked fine.

That was 3 nights ago. This evening it’s done the same thing, except repeating the steps didn’t sort it out. Running the drive through some brief SMART diagnostics and a partition check shows up no problems, so I’m inclined to believe the problem lies in the nslu2. I’ll post again if I can sort it, but at this stage I think a re-flashing is in order. Only problem is I can’t remember what firmware I flashed onto the nslu2 in the first place. D’oh!

Fix for “Current Password No Longer Matches Keyring”

A I may have mentioned, I recently reinstalled Ubuntu (had been using Windows 7 for a few months). Since the reinstall I had changed my common login password for the other computers on the network so of course I updated my user password for the new install.

The slight problem came when I used any application that accessed the keyring (say, network-manager), as there was a password mismatch between my credentials and the keyring. This was because I preserve my /home partition to keep my data (a good practice) and that is where the keyring data gets stored (logically). But it got tiresome to frequently see:

“Enter password to unlock your login keyring. The password you use to login to your computer no longer matches that of your login keyring”

It’s dead easy to fix though.

  1. Go to Applications -> Accessories -> Passwords and Encryption Keys
  2. Right-click on Passwords: login
  3. Select “Change Password”
  4. Enter your old login password (the one that it repeatedly asks for!) in the old password bit, and put your current login password in the new password fields.

And that’s all there is to it!

[Solved] sshd Does Not Run At System Startup (Ubuntu)

Problem:

  • sshd does not appear to start on system boot, but runs fine when started from a terminal with /etc/init.d/ssh start

Update Dec 2010: Thanks to Jeremie here. Change the following in /etc/init.d./ssh to stop sshd starting before the network is ready:

Change:

# Required-Start:       $remote_fs $syslog 

to:

# Required-Start:       $remote_fs $syslog $network

Merci Jeremie!

Cause and Solution:

  • A ListenServer directive in /etc/ssh/sshd_config is making sshd attempt to listen on a not-yet extant address. Change the directive to ListenServer 0.0.0.0

(NB: if you don’t have an /etc/init.d/ssh, you can get one from here)

I had a problem with a machine I am using as a samba fileserver. It would seem that the sshd process was not running at startup, so I would have to log into Gnome and run /etc/init.d/ssh start manually, which was a pain in the arse.

A quick search turned nothing up, except “make sure openssh-server is installed”, which in my case it was. I was about to post to the Ubuntu forums, but first I had a quick look at the syslog (which sshd prints to), where I saw entries like the following:


Jun 27 13:18:56 hermes init: ssh main process (802) terminated with status 255
Jun 27 13:18:56 hermes init: ssh main process ended, respawning
Jun 27 13:18:56 hermes init: ssh main process (806) terminated with status 255
Jun 27 13:18:56 hermes init: ssh main process ended, respawning
Jun 27 13:18:56 hermes init: ssh main process (810) terminated with status 255
Jun 27 13:18:56 hermes init: ssh main process ended, respawning
Jun 27 13:18:56 hermes init: ssh main process (814) terminated with status 255
Jun 27 13:18:56 hermes init: ssh main process ended, respawning
Jun 27 13:18:56 hermes init: ssh main process (818) terminated with status 255
Jun 27 13:18:56 hermes init: ssh main process ended, respawning
Jun 27 13:18:56 hermes init: ssh main process (822) terminated with status 255
Jun 27 13:18:56 hermes init: ssh main process ended, respawning
Jun 27 13:18:56 hermes init: ssh main process (826) terminated with status 255
Jun 27 13:18:56 hermes init: ssh respawning too fast, stopped

My thinking is that sshd was trying to start up before the network interfaces were configured, which was causing it to fail as it had a ListenAddress directive in /etc/ssh/sshd_config.

Commenting out the specific ListenAddress directive and adding ListenAddress 0.0.0.0 to let sshd listen on any address solved the problem. The fileserver has only 1 IP address anyway.

Black Screen (KSOD) Vista Solutions

Before I get started, some resources:

I’m looking at a friend’s laptop for him. It boots, gets past the progress bar, but before login or a desktop, the screen goes black with a cursor present which can be moved. There are multitudes of threads and posts about this issue over the internet, ranging as far back as 2006 and as recently as last month (May 1010). Although the symptoms are similar, from what I can see there are several causes each with different fixes.

For example, if the login screen comes up and there is a black screen after, it may be possible to hit Ctrl+Shit+Esc together to bring up Task Manager, from there do File->New Task (Run…), explorer.exe. This will bring up the desktop etcaetera, from where you can run other tools to repair the issue.

Advanced Startup Options Menu

Hit F8 while the computer is booting to bring up this menu. It has a number of potentially useful options, and is probably the easiest and lowest-impact to try first. The following options from that menu have been reported to work for some:

  • Safe Mode (with Networking / Command Prompt)
  • Enable low-resolution video (seems to help with nVidia driver update issues)
  • Last Known Good Configuration
  • Disable Driver Signature Enforcement (helps with some other driver errors in HP laptops, if I remember correctly)

It may also help to choose the Enable Boot Logging option, which should create a file called ntbtlog.txt containing info that might point to the driver or component that is causing issues.

Using the Windows System Recovery tool

The first option on the advanced startup menu listed above is “Repair your computer”. This starts a graphical interface with a number of options which may help.

System Restore

You should try to do a system restore first, if it is enabled. If it isn’t enabled, make sure to enable it when you fix the problem! Click the “System Restore” option on the menu (second item) and choose a restore point from when you believe your computer was previously functional.

Registry Fix

It may be a registry issue. In some cases, it would appear the Shell entry is corrupted. You can sort it by doing the following:

  1. Turn on the computer
  2. As it boots, press F8 to bring up the advanced boot options menu
  3. Select “Repair Your Computer”
  4. From there, login to an admin account (eg Administrator)
  5. Select the bottom option to launch a command prompt
  6. type regedit and hit enter
  7. Navigate through the following keys: HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\Current\Winlogon
  8. One you are at Winlogon, there should be an item on the right panel that says “Shell“. Right click this, and select edit.
  9. Change the value to explorer.exe. If this is already present, this is not your issue!

NOTE: If you try this from the “Repair Windows” option on a Windows Vista install DVD or recovery DVD it won’t work, as that seems to edit the registry on the disc (you will see cmd.exe /k start cmd.exe or similar for the Shell key). For me the “load hive” option was greyed out, if isn’t for you, you can open C:\Windows\system32\config and edit the local registry).

Startup Repair

It might be a long shot, but the startup repair option might help. It is the first option on the System Recovery menu.

Memory

It is unlikely that your RAM is causing this problem, but it is useful to know that Vista does have a tool to check memory, and it can be accessed through the System Recovery tool. I didn’t know this until today, having previously used memtest+ x86 included on Ubuntu LiveCDs.

Chkdsk / SFC (Updated 2011-07-16)

From the command prompt provided by the System Recovery tool, you can also try running chkdsk /r C: to run a disk check. Running sfc /scannow /offbootdir=c:\ /offwindir=C:\windows should do a verification of ‘critical windows files and components’, but I could not get this to run as it complained about needing to reboot with an active recovery ongoing – I guess it can’t be run via the system recovery tool.

Renaming Event Viewer Log Folder / Prefetch Folder

(See the good post by ubman on technet here)

There are reports that a problem with the event viewer service can cause this black screen on login. To work around this, start up a command prompt through the recovery tool as above. Then execute:

C:
cd \windows\system32\winevt
ren Logs LogsBad
mkdir Logs

Another report claims PreFetch can cause the issue. Now, the folder for PreFetch can be deleted as Windows will recreate its contents as needed. However, should you wish to keep a backup, start p a command prompt as above and run:

C:
cd \windows\
ren Prefetch PrefetchBad
mkdir Prefetch

Other Options

Apologies for the brevity here, these items I quickly jotted down but I am not able to try some of them myself.

Also on technet, zorglub1789 claims broken file permissions were the cause of his woes. Using icacls to fix these apparently worked. He booted to the recovery console and ran:

icacls C:\* /grant builtin\users:(RX) /T

to grant read and execute permissions to all users.

If you can start safe mode, it may be worthwhile temporarily disabling UAC. If the security centre won’t run in safe mode from the control panel, I think it is accessible via the management console. Hold the windows key + R, then type in mmc. I think you may need to add the Group Policy plugin / manager to MMC.

Another file to check on is C:\wialog.txt. This may contain useful information for troubleshooting what is causing the problem.

Again, if safe mode works, msconfig may prove useful. Hold windows key + R, and type msconfig. Then try disabling services, startup options and so forth until you get a working startup. From there you can work back to the cause of the problem. Apparently one to try is “Low Graphics Boot” (or similar), although I don’t know if there is a difference between this and the advanced startup (F8) option.

Tools

Basic Command-line Backup

If you have an external hard drive, and want to back up the C:\ drive, the following command will do that from a command prompt (see above for how to get a command prompt).
xcopy /h /i /c /k /e /r /y C: E:\backup

Where E: is the drive letter for your external hard drive.

Reset Administrator or other Windows Passwords

There is a tool at http://pogostick.net/~pnh/ntpasswd/ which is very handy for resetting Windows passwords should you not know them. I had to do it in this case as there was no documentation on the default password for this laptop (an Acer), and I can vouch for its utility. Note that you don’t need to know the password for this tool to work.

Broadcom 4318 Working Under Ubuntu Hardy Heron 8.04 (Ndiswrapper)

In my dad’s PC is a wireless card – a Linksys WRT54GS I think. Anyway it uses the Broadcom BCM 4318 chipset, as seen by a quick lspci:

richard@hades:~$ lspci | grep roadc
00:0a.0 Network controller: Broadcom Corporation BCM4318 [AirForce One 54g]
802.11g Wireless LAN Controller (rev 02)

Unfortunately the new b43 / b43-fwcutter package (installed from / by ‘restricted drivers’) stopped this working. I’m not sure why exactly; it’s possibly because it was designed for a later chipset (I’ve heard 04 for one of the chipsets), but in any case this is moot. It doesn’t work at the moment. This affects PCI cards, some internal laptop wireless cards, and some removable wireless cards.

Now, the card worked under Gutsy using ndiswrapper. So, that’s what I tried to get this working. There is a post at the Ubuntu forums that details how to do this. I’m going to briefly reproduce the procedure here because quite a lot of people are interested in getting their card woking, but thanks to Mazza558 for the info.

1) Remove the b43-fwcutter package
sudo aptitude remove b43-fwcutter

2) Reinstall ndiswrapper

a)
sudo apt-get install ndisgtk

b) Download and install wireless driver
WMP54GS Driver
or
wget http://roberthallam.com/wmp54gs.tgz
tar -xzf wmp54gs.tgz
ndiswrapper -i wirelessdriver/WMP54GS.inf

3) Create bash script to fix wireless
sudo gedit /etc/init.d/wirelessfix.sh
Into the file, put:
#!/bin/bash

modprobe -r b44
modprobe -r b43
modprobe -r b43legacy
modprobe -r ssb
modprobe -r ndiswrapper
modprobe ndiswrapper
modprobe b44

Save it, then change the permissions to 755:
cd /etc/init.d/ && sudo chmod 755 wirelessfix.sh

And finally execute:
sudo update-rc.d wirelessfix.sh defaults

And then you can reboot and have working wireless, or just (as root) execute the commands you put into the wirelessfix.sh file.

NB: To get a root bash prompt in Ubuntu, execute:

sudo bash

Enjoy your wireless, on whatever card card you have!

Update: Ed points out quite rightly that can be somewhat hard to update using an internet connection if you don’t have a working wireless card. A situation not unlike what good is a phone call… if you’re unable to speak? In any case, you have a number of options, some of which may or may not be possible:

  1. Plug a cable into your ethernet port
  2. Use an old Ubuntu 7.10 Gutsy Gibbon CD to update from. This should Just Work, that is you put it in the CD drive and you will be asked if you want to use it as a repository source (or similar). If not, the
    deb-cd
    command is your friend
  3. Download the .deb files manually elsewhere. Get both ndiswrapper-common and ndiswrapper-utils-1.9 (links to mirrors, not direct). Note that ndiswrapper-utils-1.9 has a couple dependencies like perl which you should have, but may not. Save them to a USB flash drive or whatever. Then use dpkg to install the deb files. Or:
    [insert USB drive and cd to it]
    wget
    http://mirrors.kernel.org/ubuntu/pool/main/n/ndiswrapper/ndiswrapper-common_1.50-1ubuntu1_all.deb
    wget
    http://mirrors.kernel.org/ubuntu/pool/main/n/ndiswrapper/ndiswrapper-utils-1.9_1.50-1ubuntu1_i386.deb

    [move USB drive to target machine, open a terminal and cd to the drive]
    dpkg -i ndiswrapper-common_1.50-1ubuntu1_all.deb
    dpkg -i ndiswrapper-utils-1.9_1.50-1ubuntu1_i386.deb

And you should be good to go, even if you are without an internet connection!

Irritation: Falling back to the standard locale (“C”)

Due to the pressures of university (labs, reports, and a dissertation), coupled with some of the other things I’ve been doing I haven’t posted here in ages. For now I’ll have to give you a post that is as much for you as it is for me.

Occasionally on my debian box I get an error such as:

$ perl: warning: Falling back to the standard locale ("C")

I always forget how to fix this, but the following should work:

# dpkg-reconfigure locales

If you get dpkg-reconfigure: locales is not installed, do an

apt-get install locales

and say yes when it asks you if you want to remove base-config. The functions of base-config have been superseded by locales.

You can also try using localeconf.