Friday, December 21, 2012

Extending the AD Schema on Samba4 - Part 2

Importing LDIF files into Samba4 and Active Directory

This is part 2 of the Extending AD Schema on Samba4 series.  The examples below are tested using the Samba4 LAB I created.  If you want more information on how that works then please read  http://david-latham.blogspot.co.nz/2012/12/samba4-ga-release-virtualbox-lab.html

For part one, please read http://david-latham.blogspot.co.nz/2012/12/extending-ad-schema-on-samba4.html

Unfortunately the format of an ldif file for creating new attributes and classes in the Schema Configuration are differ between Samba4 and Microsoft.

The tools are slightly different too.  So this article will attempt to make it all clear.

Find all the latest versions of code on this post at https://github.com/linuxplayground/yubikey-ldap/tree/master/microsoft-schema

Samba4 - ldbadd & ldbmodify


As far as I can tell the only way to create a new class with a custom attribute in Samba4 (on the Linux command line) is first add the attribute with ldbadd and then add the class with a seperate ldbadd followed by an ldbmodify command to update the User schema to include the new auxiliary class.

I wrote a README.md file for when this eventually ends up on GIT.  Rather than retype it all, I shall just copy it as is:

YubiKey LDIF Implimentation Into Samba4 Active Directory
=======================================================

This is an implimentation of the Openldap implimentation by Michal Ludvig <http://logix.cz/michal/devel/yubikey-ldap/> applied to Samba4 Active Directory.

CAUTION
-------
This process will permanently modify your schema.  If it breaks you will not be able to recover unless from a backup.  Please backup your schema files before starting.  On a default install they can be found at /usr/local/samba/private/sam.ldb and all the files in /usr/local/samba/private/sam.ldb.d/

yubikeyid.ldif
--------------
    dn: CN=yubiKeyId,CN=Schema,CN=Configuration,dc=samba4,dc=internal
    changetype: add
    objectClass: top
    objectClass: attributeSchema
    attributeID: 1.3.6.1.4.1.40789.2012.11.1.2.1.1
    cn: yubiKeyId
    name: yubiKeyId
    lDAPDisplayName: yubiKeyId
    description: Yubico YubiKey ID
    attributeSyntax: 2.5.5.5
    oMSyntax: 22
    isSingleValued: FALSE

Add the yubiKeyId attribute into the Schema Configuration first with:
    ldbadd -H /usr/local/samba/private/sam.ldb \
      yubikeyid.lidf \
      --option="dsdb:schema update allowed"=true

yubikeyuser.ldif
----------------
    dn: CN=yubiKeyUser,CN=Schema,CN=Configuration,dc=samba4,dc=internal
    changetype: add
    objectClass: top
    objectClass: classSchema
    governsID: 1.3.6.1.4.1.40789.2012.11.1.2.2.1
    cn: yubiKeyUser
    name: yubiKeyUser
    lDAPDisplayName: yubiKeyUser
    description: Yubico YubiKey User
    subClassOf: top
    objectClassCategory: 3
    mayContain: yubiKeyId

Next add the yubiKeyUser class into the Schema Configuration with:
    ldbadd -H /usr/local/samba/private/sam.ldb \
      yubikeyuser.lidf \
      --option="dsdb:schema update allowed"=true

updateUserClass.ldif
--------------------
    dn: CN=User,CN=Schema,CN=Configuration,DC=samba4,DC=internal
    changetype: modify
    add: auxiliaryClass
    auxiliaryClass: yubiKeyUser

Apply the User class update with:
    ldbmodify -H /usr/local/samba/private/sam.ldb \
      updateUserClass.ldif \
      --option="dsdb:schema update allowed"=true

Add YubiKeys to Users
---------------------
An example ldif:
    dn: CN=David Latham,CN=Users,DC=samba4,DC=internal
    changetype: modify
    add: objectClass
    objectClass: yubiKeyUser
    -
    add: yubiKeyId
    yubiKeyId: abcdefgh1234
    yubiKeyId: xyzxyz123456

Apply it with:
    ldapmodify -h samba -f addKeyToUser.ldif

Test it with:
    ldapsearch -h samba -b "CN=David Latham,CN=Users,DC=samba4,DC=internal" yubiKeyId

    SASL/GSSAPI authentication started
    SASL username: administrator@SAMBA4.INTERNAL
    SASL SSF: 56
    SASL data security layer installed.
    # extended LDIF
    #
    # LDAPv3
    # base with scope subtree
    # filter: (objectclass=*)
    # requesting: yubiKeyId
    #

    # David Latham, Users, samba4.internal
    dn: CN=David Latham,CN=Users,DC=samba4,DC=internal
    yubiKeyId: abcdefgh1234
    yubiKeyId: xyzxyz123456

    # search result
    search: 5
    result: 0 Success

    # numResponses: 2
    # numEntries: 1

Acknowledgments
===============
Michal Ludvig for defining the schema.
Microsoft Documentation for information on attributeSyntax, oMSyntax and objecClassCategory



Active Directory & the LDIFDE tool


Use the following ldif file and the ldifde tool as described in the README.md below:
YubiKey Implimentation in Microsft Active Directory
===================================================
This is an implimentation of the Openldap implimentation by Michal Ludvig <http://logix.cz/michal/devel/yubikey-ldap/> applied to Microsft Active Directory.

Notes
-----
In order to manage the Schema from a Windows client, please add the following line to your smb.conf under the [globals] section and restart samba4:

    dsdb:schema update allowed = true

You can use tools like ADSI Edit to manage the keys for users.
There are also tutorials on the internet explaining how to crate a dialogue box / context menu tool for updating custom attributes in the Active Directory Server Admin tool. (dsa.msc)

For a complete tutorial on all of this look at:
    <http://www.informit.com/articles/article.aspx?p=169630&seqNum=1>
   
Implimentation
--------------

Log into a Windows Server 2003 as a domain administrator and start a
command prompt.

Then execute:
    ldifde -i -f path\to\yubikey-ads.ldif -j .

You should see something like:
    6 entries modified successfully
   
    The command has completed successfully

To test if this is all working you could add some kuys using the ADSI Edit
snap-in. 
* Browse to your Domain -> CN=Users
* Right mouse click the username you want to edit
* Select Properties
* Scroll down to and select YubiKeyId
* Click Edit
* Add values until you are done
* Click OK until you are finished.


The LDIF File

#
# YubiKey LDAP schema for Microsoft Active Directory Server
#
# Install with ldifde -i -f path\to\yubikey-ads.ldif -j .
# on a Windows Command prompt
#
#
# Author: Michal Ludvig
# Consider a small PayPal donation:
#         http://logix.cz/michal/devel/yubikey-ldap/
#
# Converted to Microsoft Active Directory Server format by
#         David Latham
#
dn: CN=yubiKeyId,CN=Schema,CN=Configuration,DC=samba4,DC=internal
changetype: add
objectClass: top
objectClass: attributeSchema
cn: yubiKeyId
description: Yubico YubiKey ID
attributeID: 1.3.6.1.4.1.40789.2012.11.1.2.1.1
attributeSyntax: 2.5.5.5
isSingleValued: FALSE
oMSyntax: 22
lDAPDisplayName: yubiKeyId
name: yubiKeyId

dn:
changetype: modify
add: schemaUpdateNow
schemaUpdateNow: 1
-

dn: CN=yubiKeyUser,CN=Schema,CN=Configuration,DC=samba4,DC=internal
changetype: add
objectClass: top
objectClass: classSchema
cn: yubiKeyUser
description: Yubico YubiKey User
subClassOf: top
governsID: 1.3.6.1.4.1.40789.2012.11.1.2.2.1
mayContain: yubiKeyId
rDNAttID: cn
objectClassCategory: 3
lDAPDisplayName: yubiKeyUser
name: yubiKeyUser

dn:
changetype: modify
add: schemaUpdateNow
schemaUpdateNow: 1
-

dn: CN=User,CN=Schema,CN=Configuration,DC=samba4,DC=internal
changetype: modify
add: auxiliaryClass
auxiliaryClass: yubiKeyUser
-

dn:
changetype: modify
add: schemaUpdateNow
schemaUpdateNow: 1
-

Extending the AD Schema on Samba4 - Part 1

My last post on Samba4 showed how easy it is to install and configure an AD Service on Linux.  If you've not read it then please have a look. (http://david-latham.blogspot.co.nz/2012/12/samba4-ga-release-virtualbox-lab.html)

This post show's how to extend the Samba4 Active Directory Schema.  Specifically for YubiKey integration.

YubiKey's can be purchased for a relativlely low price from Yubico.  Please visit their website (www.yubico.com) for more information.

LDAP Integration is very well covered by Michal Ludvig on his website and github.  (http://www.logix.cz/michal/devel/yubikey-ldap/) In fact we are planning to leverage his implementation at our work and are considering donating towards what's obviously a very good cause.

Now seeing as though LDAP and AD are so similar and exhibit many of the same APIs, I began to wonder how this might fit in with Samba4.  Eventually we might end up using Samba4 for our domain and so I needed to figure out if I could, somehow, "port" the LDAP Schemas for Yubikey to AD.  I think I have managed it with extensive help from another blog.   Thanks to Kurt Hudson and his wonderful article on www.informit.com titled, "Making AD Work Harder | Extending the Active Directory Schema to Track Custom Info"

I should also acknowledge, once again, the Samba Wiki for a crucial piece of the puzzle in which we allow extensions to the Schema. (https://wiki.samba.org/index.php/Samba4/Schema_extenstions)

Step 1: Backup Your Master Server

In my case this was fairly simple.  I just took a snapshot of my Virtualbox image.  You need to do this though.  Messing with an AD schema can be fairly dangerous, especially if it's a Samba4 AD.  Samba's Wiki makes this quite clear.

Step 2: Enable Schema Extensions

This is fairly straight forward and well explained on the Samba Wiki.  First edit your smb.conf and add the following line into the [globals] section.
dsdb:schema update allowed = true
Then restart the samba service.. (btw: If you are interested in an init script for Samba4 on Centos 6 then find it at the end of this article.)

Step 3: Set up Active Directory Schema Console (on Windows)

From here on, we will be working in Windows space.  The Active Directory Schema console is not registered by default.  So register it and create yourself a console.

In a windows command prompt type:
regsvr32 schmmgmt.dll

Then use MMC -> Add Remove Snap-ins ->  Active Directory Schema snap-in. (follow the instructions on Kurt Hudson's page.

Step 4: Add Yubikey Class and Attribute

You need an attribute before you can create an object class.
The details for the Yubikey schema are available on Michal Ludvig's github site: https://raw.github.com/mludvig/yubikey-ldap/master/ldap-schema/yubikey-SunDS.ldif
The SunDS describes most closely the AD Version used in this article:
Here it is:

dn: cn=schema
objectclass: top
#
# YubiKey LDAP schema for Sun Directory Server and OpenDJ
#
# Copy this file to /config/schema/05-yubikey.ldif
#
# Author: Michal Ludvig 
# Consider a small PayPal donation:
#         http://logix.cz/michal/devel/yubikey-ldap/
#
# Converted for OpenDJ and Sun Directory Server by
#         Ludovic Poitou 
#
attributeTypes: ( 1.3.6.1.4.1.40789.2012.11.1.2.1.1
  NAME 'yubiKeyId'
  DESC 'Yubico YubiKey ID'
  EQUALITY caseIgnoreIA5Match
  SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{128} )
objectClasses: ( 1.3.6.1.4.1.40789.2012.11.1.2.2.1
  NAME 'yubiKeyUser'
  DESC 'Yubico YubiKey User'
  SUP top
  AUXILIARY
  MAY ( yubiKeyId ) )

Step 4a: Attribute

In the Active Directory Schema Snap-in Console do the following:
  • Right click the Attributes Node and select Create Attribute on the context menu.  A message about the permanence of your operation will show up.  Well, jump right in.  You backed up in step 1.
  • Fill in the details as follows:
    • Common Name = yubiKeyId
    • LDAP Display Name = yubiKeyId
    • Unique X500 Object ID:  = 1.3.6.1.4.1.40789.2012.11.1.2.1.1
    • Description = Yubico YubiKey User
    • Syntax = Case Insensitive String
    • Minimum = 0
    • Maximum = 128
    • Multi Valued = Ticked
  • Click OK to add the attribute to the Attributes list.  If you expand the list, and scroll to the bottom, you will find yubiKeyId and you can right click, select properties and see something like this:
Note, I ticked "multi valued" but in the part where I create a dialogue box to enter the key id's I do not allow for multiple values.  That's on my TO DO List.

Step 4b: Add Yubikey Schema Object Class

  • Begin by right mouse clicking Classes and select Create Class in the context menu.
  • Fill in the details as follows:
    • Common Name: yubiKeyUser
    • LDAP Display Name: yubiKeyUser
    • Unique X500 Object ID:  1.3.6.1.4.1.40789.2012.11.1.2.2.1
    • Description: Yubico YubiKey User
    • Parent Class: top
    • Class Type: Select Auxilary
  • Click Next
    • Click Add next to Opional:
    • Scroll all the way to the bottom and select yubiKeyId
    • Click OK
    • Click Finish
  • Expand the Classes node, scroll to the bottom and right click yubiKeyUser, select Properties to see the following:



So that's the GUI method.
My next post will show how to import ldif files to make this process easier.

Monday, December 17, 2012

Samba4 GA Release - Virtualbox LAB

Introduction

Configuring Samba has always been a pain in the you-know-what.  There is always some kind of permission here or there missing or misconfiguration resulting in endless hours of log trawling and frustrated users.
My profile photo on this blog was taken during just such a time...

So I decided to build a LAB for Samba 4.  My LAB is for a fresh install.  I have not yet tried an upgrade.  Maybe that will come in a later post.

The Samba4 how-to on their WIKI is very good and formed the basis of most of the work.  (http://wiki.samba.org/index.php/Samba4/HOWTO)

Lab Environment

  • LAB Built on Virtual Box using Ubuntu 12.04 LTS
  • Samba Server
  • Client
    • Windows2003 R2 Standard Edition

Samba OS Build

Hard Disk Configuration

The Samba HOW-TO wiki states that some specific mount attributes are applied to your disks.  Here is my fstab.
[root@samba ~]# cat /etc/fstab

#
# /etc/fstab
# Created by anaconda on Thu Dec 13 13:55:41 2012
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/VolGroup-lv_root /            ext4 user_xattr,acl,barrier=1 1 1
ther "system" lines not shown here>
# SAMBA
/dev/mapper/vgsamba-lvwinhome /samba/home ext4 user_xattr,acl,barrier=1 1 1

Network Interface Configuration

  • /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=samba.example.com
  • /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE="eth0"
NM_CONTROLLED="no"
ONBOOT="yes"
BOOTPROTO="dhcp"
DHCP_HOSTNAME=samba.vbox.local
  • /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE="eth1"
NM_CONTROLLED="no"
ONBOOT="yes"
BOOTPROTO="none"
IPADDR="10.1.1.1"
NETMASK="255.255.255.0"

Iptables

For Iptables, I went with an open configuration.  You should have a good and proper IPTABLES configuration in your production environment.  This LAB is about SAMBA4 not IPTABLES.
  • /etc/sysconfig/iptables
    • The IPTABLES script below impliments a SNAT on all traffic to the IP Address of eth0.  Your IP address might be different.  This all depends on how you configured your host only network in Virtualbox.
# Generated by iptables-save v1.4.7 on Thu Dec 13 14:26:37 2012
*nat
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A POSTROUTING -o eth0 -j SNAT --to-source 192.168.56.76

COMMIT
# Completed on Thu Dec 13 14:26:37 2012
# Generated by iptables-save v1.4.7 on Thu Dec 13 14:26:37 2012
*filter
:INPUT ACCEPT [12:1634]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT
# Completed on Thu Dec 13 14:26:37 2012

Enable IPv4 Packet Forwarding

Once you do this, routing will take care of itself.
  • Set net.ipv4.ip_forward = 1 in /etc/sysctl.conf
  • execute
    • sysctl -p to read the new value.

Software pre-requisites and services

Software packages

First establish the following dependancies.  (Note I used the groupinstall "Development tools" for convenience.

Read: http://wiki.samba.org/index.php/Samba_4/OS_Requirements
  • yum -y install libacl-devel libblkid-devel gnutls-devel readline-devel python-devel gdb pkgconfig krb5-workstation zlib-devel setroubleshoot-server setroubleshoot-plugins policycoreutils-python libsemanage-python setools-libs-python setools-libs popt-devel libpcap-devel sqlite-devel libidn-devel libxml2-devel libacl-devel libsepol-devel libattr-devel keyutils-libs-devel cyrus-sasl-devel openssh-clients bind-utils dhcpd tcpdump man manpages wget
  • yum -y groupinstall 'Development tools'

NTP

  • Download and install the latest NTP.
# Start NTP Daemon
/sbin/ntpd -c /etc/ntp.conf

DHCP Server

  •  Configure dhcpd to provide ip addresses on the internal network.
    • dhcpd should also provide the netbios and wins ip addresses.
 subnet 10.1.1.0 netmask 255.255.255.0 {
        option routers          10.1.1.1;
        option subnet-mask      255.255.255.0;
        option domain-name      "internal.local";
        option domain-name-servers 10.1.1.1;
        option netbios-name-servers 10.1.1.1;
        range 10.1.1.10 10.1.1.100;
}
    • Edit /etc/sysconfig/dhcpd and set DHCPDARGS=eth1
    • service start dhcpd
    • chkconfig dhcpd on

DNS Server

I did not configure bind or dnsmasq for the Domain.  Samba4 Ships with an internal DNS server which is adequate for the purposes of demonstrating Samba4 awesomness.  The excellent BIND Nameserver is supported by Samba4 along with dns replication and all that jazz.  I have not tried it.  If you have, let us know in the comments below how it went.

Once Samba is installed and operational there is a step required to make Samba forward DNS queries to your host's network as well as make the OS resolv.conf use the correct DNS.  This is because dhclient likes to automagially configure your resolv.conf every time you boot.  My work-around is just to overwrite it in /etc/rc.local.

# fix resolv.conf
mv /etc/resolv.conf /etc/resolv.conf.old
cat >> /etc/resolv.conf << EOF
# Updated by rc.local script
domain samba4.internal
nameserver 10.1.1.1
EOF


Install Samba4

Installing Samba4 is fairly straightforward once you have all the pre-requisites in place.  It's well documented on the samba4 wiki so I won't get into it in any great detail here.

I highly recommend completing the file system and kerberos tests outlined in the wiki how-to. (http://wiki.samba.org/index.php/Samba4/HOWTO#Step_6:_Testing_Samba4)

smb.conf

This is my lab smb.conf.

  • workgroups, realm, netbios and server role all define the function of this samba instance.
  •  dns forwarder tells samba where to send dns requests for hosts it knows nothing about.
  • The netlogon share is for the logon script.  I have a log on script located in it's path and I set the default logon script in windows for each user to point to this script.
  • sysvol is (unused at this stage)
  • home is where all the user's WINDOWS home directories live
    • There are no unix home directories that map to the windows home directories
  • share is where all standard shares will live.


[root@samba ~]# cd /usr/local/samba/etc
[root@samba etc]# cat smb.conf
# Global parameters
[global]
        workgroup = SAMBA4
        realm = SAMBA4.INTERNAL
        netbios name = SAMBA
        server role = active directory domain controller
        dns forwarder = 192.168.56.1

[netlogon]
        path = /usr/local/samba/var/locks/sysvol/samba4.internal/scripts
        read only = No

[sysvol]
        path = /usr/local/samba/var/locks/sysvol
        read only = No
[home]
        comment = Home Directories
        path = /samba/home
        read only = No
[share]
        comment = Office Shares
        path = /samba/home/shares
        read only = No
[root@samba etc]#


Configuring File Permissions and Shares

Once these shares were configured in this basic way using smb.conf I was able to browse to the folders in my windows client using as follows:
\\samba\home
\\samba\share
\\samba\netlogon\logon.bat
\\samba\sysvol

This was easily the most difficult part of the process for me to work out.  Once worked out though, it's a piece of cake.  Critically your file system must be mounted with the acl mount option.

I have figured out that the most straight forward way of managing file shares is via Windows.  ie: Let samba take care of the acl entries in Linux.

User accounts and home directories

Home directory ACLs

Before creating user accounts its critical to make sure that the windows ACLs are configured correctly for \\samba\home.  Mine are as follows:

First remove inheritance.  IE: The permissions for \\samba\home must not inherit from \\samba.  When removing inheritance hit the remove button and remove any other permissions groups / users listed.

Then add new ones as follows:
  • Administrators = Full Control
  • Everyone = No Control (all boxes unticked)
  • Domain Users = Special Permissions as follows
    • Traverse Folder and Execute File = TICKED
    • All other special permissions = UNTICKED

Creating new users

Run the dsa.msc snap in.  Installation instructions for the dsa.msc are covered in the samba4 wiki howto.

Create a new user using the "New -> User" task in the DSA Snapin.  The following screenshots show what to do:

Dialogue box showing how to add a user with teh Windows DSA.  First, Last and user name
Add user Test User with username test.user

Dialogue box showing how to add a user with teh Windows DSA.  Password
Set a password

Dialogue box showing how to add a user with teh Windows DSA.  Confirmation
Confirm details
Dialogue box showing how to configure the user profile in the Windows DSA.  Set the logon path and the home folder
Bring up the Profile tab under properties for this new user.


Here, on the profile tab you should set the logon script for your user and map the home drive on the connect  button.  When you apply this, the test.user folder will be created on the share for you.  The permissions will also be configured appropriately as per the next screenshot.

Windows Security permissions dialogue box for new user home folder.
Confirm folder security permissions are correct.


Use a similar approach to managing shared folders for group access.

So far I have not encountered any gotchas with this approach.


Saturday, October 13, 2012

Oracle Cloud Control

I had the occasion recently to deploy an OVM stack at a customer site.  Initially it was my intention to only deploy Enterprise Manager 12c and take advantage of the cloud control features within.  As it turned out, this provides merely a "remote control" function of an existing OVM Manager.  With this in mind, I could not find many advantages to using EM12c for Oracle virtual machine management.

The next hurdle was with the hardware.  Of course, the physical servers we deployed the OVS 3.1.1. hyper-visor on were not "Oracle Certified."  Specifically DELL PowerEdge M620 Blade Servers.  (http://www.dell.com/us/enterprise/p/poweredge-m620/pd?~ck=anav)

The customer had selected the Broadcom®  57810S-k Dual Port 10Gb KR blade NDC on board Network Adapter as well as the Mezzanine Broadcom 5719 Serdes Quad Port 1Gb providing a total of 6 ethernet ports, two of them 10GB.

The storage array is Equalogic iSCSI SAN.  (Sorry the exact make / model of the SAN escapes me right now.)

The OVM 3.1.1. Server installation went perfectly well.  We selected the 3rd port for management (1st port on the 1g mezzanine card) knowing that ports 1 and 2 were to be used for iSCSI and possibly Live Migration.

The blades were able to be discovered without difficulty into OVM manager.  Apon inspection, however, eth0 was showing as DOWN in all three blade servers.  The iDRAC console also showed that the link was UP but the OS State was reported as DOWN in each case.  I was able to successfully configure eth1 and create a single port bond with the appropriate IP address for the SAN and discover the storage.  Just without redundancy at this stage.  Of course this had to be resolved before production.

I did some research and discovered that the OEM 3.1.1. hyper-visor is based on RHEL / OL 5.7 and that the broadcom driver (bnx2x) needed an option to force all ports back to legacy state.

From: ftp://ftp.supermicro.com/CDR-X9_1.11_for_Intel_X9_platform/Broadcom/Server/Linux/Driver/README.bnx2x.TXT

Driver Parameters
=================

Several optional parameters can be supplied as a command line argument
to the insmod or modprobe command. These parameters can also be set in
modprobe.conf. See the man page for more information.

The optional parameter "int_mode" is used to force using an interrupt mode
other than MSI-X. By default, the driver will try to enable MSI-X if it is
supported by the kernel. In case MSI-X is not attainable, the driver will try
to enable MSI if it is supported by the kernel. In case MSI is not attainable,
the driver will use legacy INTx mode. In some old kernels, it's impossible to
use MSI if device has used MSI-X before and impossible to use MSI-X if device
has used MSI before, in these cases system reboot in between is required.

Set the "int_mode" parameter to 1 as shown below to force using the legacy
INTx mode on all NetXtreme II NICs in the system.

   insmod bnx2x.ko int_mode=1

or

   modprobe bnx2x int_mode=1


The way I achieved this was to add a file into /etc/modprobe.d as follows:

/etc/modprobe.d/bxn2x.conf
options bxn2x int_mode=1

Reboot the blades and viola, eth0 shows status UP, ethtool eth0 indicates 10gb speed on the port and the iDRAC console shows all good on the port.
I then added eth0 to the bond I created earlier and left it in Active / Backup configuration.

I was unable to make multipathing work through the GUI.  I suspect that OVM manager only discovers a single path to storage and does not attempt anything further.  I am confident that the Active Backup bond configuration will provide sufficient storage redundancy for the storage pool, server pool repository and PVM Guest LUNS.

I hope this helps anyone who is also struggling with this issue.

Monday, May 14, 2012

Oracle Virtual Machine (OVM) LAB on VirtualBox

Oracle Virtual Machine (OVM) 3.1.1. was released on the 8th May 2012 and finally it's supported on VirtualBox.  This is great news for anyone who wants to give installing an OVM Lab a go.

I gave it a go.

I created 3 VirtualBox guests:
  1. Openfiler for iSCSI with a 40GB virtual disk for chopping up into LUNs and iSCSI.
  2. Oracle Linux 6.0 on which I installed OVM 3.1.1. in Demo mode.  This guest has 4GB of RAM assigned to it and a 25GB HD.  Probably a bit big, but OVM is a large application with an Oracle XE database and Oracle Weblogic services installed, I thought better safe than sorry.
  3. Oracle Virtual Server 3.1.1.  This guest had just a 4GB HD and 1536MB of RAM which I figured would be just enough to get 1 virtual machine up and running on it.
My VirtualBox is configured with a couple of host only networks:

vboxnet0 => 192.168.56.0/24 => Management network.  I also have dnsmasq configured on this network to serve IP addresses via DHCP to clients on this network.  Dnsmasq is installed on my laptop for this specific purpose.  See previous blog post...
vboxnet1 => 192.168.57.0/24 => Used for the storage network.
vboxnet2 => 192.168.58.0/24 => Production network for virtual machines.

So I set up openfiler to offer 2x8GB LUNS, 1x2GB LUN for the cluster storage network on my server pool and 10GB NFS Share for the repository.

The installation of OVS and OVM on Oracle Linux all went by without a hitch.  I made sure that each VirtualBox guest set their own hostnames against dnsmasq using the DHCP_HOSTNAME parameter on /etc/sysconfig/network.  This meant that all guests resolve nicely on DNS which is a prerequisite for a successful OVM LAB.

The new OVM3.1.1. GUI is actually quite nice.  It's more polished and intuitive than 3.0.3. and more of the Right Mouse Button menus are enabled.

I also made sure that SELINUX and IPTABLES was disabled in both OVS and the OVM guests.

Server discovery went fine.  No problems at all.  Just enter the agent password and hostname of the OVS server.

Storage discovery was a little tricky.  The NFS Share (Filesystem share) was no problem.  OVM was able to find and mount it wouthout any difficulty.  The only trouble I had there was making sure that Openfiler was configured correctly.  But that's not relevant to the OVM LAB really.

The iSCSI discovery had me stumped for quite some time.  I could discover the iSCSI target presented by Openfiler without issue but could not get OVM to show me any LUNS. (Physical Disks)  After much trial and error I finally figured out that my OVS server did not have the multipathd service running.  Once I enabled and started multipathd (no change to default configuration), I was able to refresh the iSCSI storage in OVM and the physical disks were presented.  To recap: First perform iSCSI storage discovery using the OVS guest as the Storage Manager.  Then start multipathd on the OVS guest, then refresh the iSCSI storage on OVM and your LUNs will appear.

I was able to create a server pool, assign the repository, assign the 2GB iSCSI LUN to the pool for it's cluster configuration data and actually configure a virtual machine.  I configured an Ubuntu VM with just 512MB of ram and one iSCSI LUN.

Finally after so much success, my LAB failed me at the most crucial stage.  Starting the virtual machine. - It would not start.  It turns out that VirtualBox does not present hardware acceleration to it's guests.  So my OVS guest was running without any hardware acceleration in the virtual CPU.  OVM was not able to start the VM.  What a shame but there  you go.

The good news is that many of the minor irritations in OVM3.0.3 seem to be cleared up and folks are able to now use VirtualBox for their LABs but they can't start OVM Virtual Machines if their OVS is actually a VirtualBox guest.

Next step... Commandeer an old server somewhere, install OVS, bind it to the right LAN and carry on.

Wednesday, May 9, 2012

Create a NAT for VirtualBox Host Only Network

VirtualBox does a fairly good job with the built in NAT feature on the virtual network device. But there are some drawbacks:
  1.  You can't easily monitor the network on the built in NAT.  So if your VM is misbehaving, you couldn't use tcpdump to troubleshoot.
  2. The built in NAT reaches directly to the Internet in a transparent manner.  If you wanted to control access to the Internet you would have to switch to a bridge device or go with host-only.
  3. Most importantly for me, it does not support GRE packets very well.  This means that if you want to access a Microsoft PPTP VPN from your VM, it won't work with the VirtualBox built in NAT. (Well it didn't for me anyway)
EDIT: An updated script for the firewall component is described in a new post found here: http://david-latham.blogspot.co.nz/2013/03/firewall-script-for-opensuse-and-others.html

So with some simple tweaks to your host, you can NAT the VirtualBox host only network and enjoy some extra features.

In VirtualBox create a default host-only network.  This is done in File -> Preferences -> Network.  Click the green icon with a PLUS sign in it.  A new network is created called vboxnet0.  Then click the little screwdriver icon to edit the vboxnet0 settings.  Select the DHCP Server tab and make sure "Enable Server" is unticked.  It will be ticked by default.

Make sure that the DHCP server is not enabled.  This is important because you will configure DHCP with dnsmasq on the next step.

Then install dnsmasq.  You need this for DNS.  I like that with dnsmasq, and dhcp client id's specified on the VM's, I can do things like "ssh vm_hostname" from the host pc.

sudo apt-get install dnsmasq

Accept the defaults.

dnsmasq uses a config.d approach for configuration so create a file called: /etc/dnsmasq.d/virtualbox as follows:

# cat /etc/dnsmasq.d/virtualbox
interface=vboxnet0
dhcp-range=192.168.56.2,192.168.56.150


Restart dnsmasq:

sudo /etc/init.d/dnsmasq restart

Now configure IPTABLES.  I took a lazy approach and added these commands to /etc/rc.local
I have not bothered to learn how to do this with ubunut's default UFW becasue I don't much like UFW and so don't use it.

sudo ufw disable (if you're interested ...)

Add the following lines to /etc/rc.local (before the last line which is "exit 0")
# Flush IPTABLES
iptables -F
iptables -t nat -F
# Loopack
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

# Allow all from virtualbox
iptables -A INPUT -s 192.168.56.0/24 -j ACCEPT

# Accept SSH  If you have openssh installed that is.
iptables -A INPUT -p tcp --dport 22 -m state --state NEW,ESTABLISHED -j ACCEPT

# Accept DNS
#iptables -A INPUT -p tcp --dport 53 -j ACCEPT
#iptables -A INPUT -p udp --dport 53 -j ACCEPT

# Accept DHCP
#iptables -A INPUT -p tcp --dport 67 -j ACCEPT
#iptables -A INPUT -p udp --dport 67 -j ACCEPT

# Masquerade virtualbox network
iptables -t nat -A POSTROUTING -s 192.168.56.0/24 -j MASQUERADE

#turn on ip forwarding.  Can be done in /etc/syctl.conf - As I said - lazy.
sysctl -w net.ipv4.ip_forward=1

Then reboot your computer.

sudo reboot

Now change the nic on your VirtualBox vm to a Host Only network and start the VM.  If you did everything right, your VM will start, obtain a lease from dnsmasq and have access to the internet.

The firewall rules above allow everything from VirtualBox to go out on the internet.  If you wanted to lock things down a bit, you could add more rules to do so and remove the line that accepts everything from the 192.168.56.0/24 subnet.

Oh and your Microsoft PPTP VPN will work now.  Even if you use wireless...

Have fun!

Sunday, April 29, 2012

Parsing the Multipath

Having worked a little with XIV I have come to like the "xiv_devlist" tool.  "xiv_devlist" lists all the attached disks on the SAN and the number of paths that are currently active.  This is very useful when you want to ensure that all the paths to storage are active.

Find latest up-to-date code for this post on my Github profile: https://github.com/linuxplayground/mpath_devlist

It surprised me to find that there are no useful tools to parse the output from Multipath.

Here is an example:

multipath -ll
mpath2 (3600507680191014a3800000000000100) dm-7 IBM,2145                       
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]                       
\_ round-robin 0 [prio=100][active]                                            
 \_ 0:0:1:8  sdaa 65:160 [active][ready]                                       
 \_ 1:0:1:8  sdbk 67:224 [active][ready]                                       
\_ round-robin 0 [prio=20][enabled]                                            
 \_ 1:0:0:8  sdas 66:192 [active][ready]                                       
 \_ 0:0:0:8  sdi  8:128  [active][ready]                                       
mpath0 (3600507680191014a380000000000010a) dm-17 IBM,2145                      
[size=50G][features=1 queue_if_no_path][hwhandler=0][rw]                       
\_ round-robin 0 [prio=100][active]                                            
 \_ 1:0:1:0  sdbc 67:96  [active][ready]                                       
 \_ 0:0:1:0  sds  65:32  [active][ready]                                       
\_ round-robin 0 [prio=20][enabled]                                            
 \_ 1:0:0:0  sdak 66:64  [active][ready]                                       
 \_ 0:0:0:0  sda  8:0    [active][ready]                                       
mpath18 (3600507680191014a38000000000000ff) dm-24 IBM,2145                     
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]                       
\_ round-robin 0 [prio=100][active]                                            
 \_ 1:0:0:7  sdar 66:176 [active][ready]                                       
 \_ 0:0:0:7  sdh  8:112  [active][ready]                                       
\_ round-robin 0 [prio=20][enabled]                                            
 \_ 1:0:1:7  sdbj 67:208 [active][ready]                                       
 \_ 0:0:1:7  sdz  65:144 [active][ready]                                       
mpath17 (3600507680191014a38000000000000fe) dm-23 IBM,2145                     
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]                       
\_ round-robin 0 [prio=100][active]
 \_ 1:0:0:6  sdaq 66:160 [active][ready]
 \_ 0:0:0:6  sdg  8:96   [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 1:0:1:6  sdbi 67:192 [active][ready]
 \_ 0:0:1:6  sdy  65:128 [active][ready]
mpath16 (3600507680191014a3800000000000109) dm-22 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
 \_ 1:0:0:5  sdap 66:144 [active][ready]
 \_ 0:0:0:5  sdf  8:80   [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 1:0:1:5  sdbh 67:176 [active][ready]
 \_ 0:0:1:5  sdx  65:112 [active][ready]
mpath9 (3600507680191014a380000000000010c) dm-14 IBM,2145
[size=1.0G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
 \_ 0:0:1:15 sdah 66:16  [active][ready]
 \_ 1:0:1:15 sdbr 68:80  [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 1:0:0:15 sdaz 67:48  [active][ready]
 \_ 0:0:0:15 sdp  8:240  [active][ready]
mpath15 (3600507680191014a3800000000000108) dm-21 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
 \_ 1:0:1:4  sdbg 67:160 [active][ready]
 \_ 0:0:1:4  sdw  65:96  [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 1:0:0:4  sdao 66:128 [active][ready]
 \_ 0:0:0:4  sde  8:64   [active][ready]
mpath8 (3600507680191014a380000000000010b) dm-13 IBM,2145
[size=1.0G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
 \_ 1:0:0:14 sday 67:32  [active][ready]
 \_ 0:0:0:14 sdo  8:224  [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 0:0:1:14 sdag 66:0   [active][ready]
 \_ 1:0:1:14 sdbq 68:64  [active][ready]
mpath14 (3600507680191014a3800000000000107) dm-20 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
 \_ 1:0:0:3  sdan 66:112 [active][ready]
 \_ 0:0:0:3  sdd  8:48   [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 1:0:1:3  sdbf 67:144 [active][ready]
 \_ 0:0:1:3  sdv  65:80  [active][ready]
mpath7 (3600507680191014a3800000000000105) dm-12 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
 \_ 1:0:0:13 sdax 67:16  [active][ready]
 \_ 0:0:0:13 sdn  8:208  [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 0:0:1:13 sdaf 65:240 [active][ready]
 \_ 1:0:1:13 sdbp 68:48  [active][ready]
mpath13 (3600507680191014a3800000000000106) dm-19 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
 \_ 1:0:1:2  sdbe 67:128 [active][ready]
 \_ 0:0:1:2  sdu  65:64  [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 1:0:0:2  sdam 66:96  [active][ready]
 \_ 0:0:0:2  sdc  8:32   [active][ready]
mpath6 (3600507680191014a3800000000000104) dm-11 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
 \_ 0:0:1:12 sdae 65:224 [active][ready]
 \_ 1:0:1:12 sdbo 68:32  [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 1:0:0:12 sdaw 67:0   [active][ready]
 \_ 0:0:0:12 sdm  8:192  [active][ready]
mpath12 (3600507680191014a38000000000000fd) dm-18 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
 \_ 1:0:0:1  sdal 66:80  [active][ready]
 \_ 0:0:0:1  sdb  8:16   [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 1:0:1:1  sdbd 67:112 [active][ready]
 \_ 0:0:1:1  sdt  65:48  [active][ready]
mpath5 (3600507680191014a3800000000000103) dm-10 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
 \_ 1:0:0:11 sdav 66:240 [active][ready]
 \_ 0:0:0:11 sdl  8:176  [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 0:0:1:11 sdad 65:208 [active][ready]
 \_ 1:0:1:11 sdbn 68:16  [active][ready]
mpath11 (3600507680191014a380000000000010e) dm-16 IBM,2145
[size=1.0G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
 \_ 0:0:1:17 sdaj 66:48  [active][ready]
 \_ 1:0:1:17 sdbt 68:112 [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 1:0:0:17 sdbb 67:80  [active][ready]
 \_ 0:0:0:17 sdr  65:16  [active][ready]
mpath4 (3600507680191014a3800000000000102) dm-9 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
 \_ 0:0:1:10 sdac 65:192 [active][ready]
 \_ 1:0:1:10 sdbm 68:0   [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 1:0:0:10 sdau 66:224 [active][ready]
 \_ 0:0:0:10 sdk  8:160  [active][ready]
mpath10 (3600507680191014a380000000000010d) dm-15 IBM,2145
[size=1.0G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
 \_ 1:0:0:16 sdba 67:64  [active][ready]
 \_ 0:0:0:16 sdq  65:0   [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 0:0:1:16 sdai 66:32  [active][ready]
 \_ 1:0:1:16 sdbs 68:96  [active][ready]
mpath3 (3600507680191014a3800000000000101) dm-8 IBM,2145
[size=40G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
 \_ 1:0:0:9  sdat 66:208 [active][ready]
 \_ 0:0:0:9  sdj  8:144  [active][ready]
\_ round-robin 0 [prio=20][enabled]
 \_ 0:0:1:9  sdab 65:176 [active][ready]
 \_ 1:0:1:9  sdbl 67:240 [active][ready]

So if you wanted to check that there are 4 active / ready paths to each "mpath" disk, you would have to visually scan through the whole output above.

A simple perl script that takes the output from multipath -ll as its input, can make this task much simpler.

dave@davidpc:~/devel/paths$ cat paths.txt | ./mpath_list.pl 
00      PATH    UID                                     DM-ID   ACTIVE  RESULT
2       mpath2  3600507680191014a3800000000000100       dm-7    4 of 4  OK
0       mpath0  3600507680191014a380000000000010a       dm-17   4 of 4  OK
18      mpath18 3600507680191014a38000000000000ff       dm-24   4 of 4  OK
17      mpath17 3600507680191014a38000000000000fe       dm-23   4 of 4  OK
16      mpath16 3600507680191014a3800000000000109       dm-22   4 of 4  OK
9       mpath9  3600507680191014a380000000000010c       dm-14   4 of 4  OK
15      mpath15 3600507680191014a3800000000000108       dm-21   4 of 4  OK
8       mpath8  3600507680191014a380000000000010b       dm-13   4 of 4  OK
14      mpath14 3600507680191014a3800000000000107       dm-20   4 of 4  OK
7       mpath7  3600507680191014a3800000000000105       dm-12   4 of 4  OK
13      mpath13 3600507680191014a3800000000000106       dm-19   4 of 4  OK
6       mpath6  3600507680191014a3800000000000104       dm-11   4 of 4  OK
12      mpath12 3600507680191014a38000000000000fd       dm-18   4 of 4  OK
5       mpath5  3600507680191014a3800000000000103       dm-10   4 of 4  OK
11      mpath11 3600507680191014a380000000000010e       dm-16   4 of 4  OK
4       mpath4  3600507680191014a3800000000000102       dm-9    4 of 4  OK
10      mpath10 3600507680191014a380000000000010d       dm-15   4 of 4  OK
3       mpath3  3600507680191014a3800000000000101       dm-8    4 of 4  OK

And the perl script that creates this output:
EDIT: Revised script (2012-04-29) Added size, Removed flag variable from script.
#!/usr/bin/env perl
use Term::ANSIColor;

$count = 0;
print "00\tPATH\tUID\t\t\t\t\tDM-ID\tSIZE\t\tACTIVE\tRESULT\n";
while(<STDIN>) {
        my ($line) = $_;
        chomp($line);
        if( $line =~ /^mpath(\d+)/ ) {
                my( $id ) = $1;
                /^(\w*) \((\w*)\)\s*(dm-\d*)/;
                if( $count > 0 ) { print $count, " of 4\t";
                        if( $count == 4) {
                                print  color 'bold green'; print "OK\n"; print color 'reset';
                        } else {
                                print color 'bold red'; print "PROBLEM\n"; print color 'reset';
                        }
                        $count = 0;
                }
                my ($pname) = $1;
                my ($puid)  = $2;
                my ($pdmid) = $3;
                print $id,"\t",$pname,"\t",$puid,"\t",$pdmid,"\t";
        }
        if ( $line =~ m/^\[(.*?)\]/ ) {
                print $1,"\t";
        }
        if ( $line =~ /^\s\\_.*active\]\[ready/ ) {
                $count += 1;
        }
}
if( $count > 0 ) { print $count, " of 4\t";
        if( $count == 4) {
                print  color 'bold green'; print "OK\n"; print color 'reset';
        } else {
                print color 'bold red'; print "PROBLEM\n"; print color 'reset';
        }

        $count = 0;
}
print color 'reset';
exit 0;

TO DO:
  1. Call "multipath -ll" from inside the perl script thus remove the requirement to pipe the results of multipath -ll into the script.
  2. Make sure multipath -ll is a valid system call.
  3. Add a summary of the results, IE: 18 paths found, ALL GOOD.
  4. Add a check to ensure user is root at the beginning of the script and exit with message if not.
  5. Sort output results in order of mpath number.

Sunday, April 15, 2012

Policing my network

Policing my network - well not actually policing it, just locking it down.

I have decided it's time to worry about what my kids might find on the Internet - my oldest is approaching 7 and beginning to spend more time on the 'puter.

So here is the plan.  Not completely implemented just yet but getting there.

  1. Enable mac address filtering on the router.  Only devices I know about are allowed to connect to the wireless router and a password is still required to gain access and an IP address on my LAN.  - DONE!  It works great and a quick visit to my router admin page shows me who is currently using it.   Look this is just good sense.  If you are not doing mac address filtering book-mark this blog right now, and go and configure your router!
  2. Have a conversation with the child about the dangers and pitfalls of the Internet.  NOT DONE YET!  - I am kind of dreading this bit.  How do I explain to a child that it's bad to do something in such a way that they don't immediately go and do it.  (It's what I would have done if my parents warned me about porn on the Internet and then gave me access to the Internet.)  Kind of like God telling Adam and Eve not to eat the forbidden fruit then showing them where it is...
  3. Create an account with openDNS (http://www.opendns.com/) - start off with the free one.  - DONE!  Testing at the moment on my own PC.  Tried visiting a couple of porn sites I remember from my bachelor days and yes - openDNS is blocking them.   ----  I Should note, however, that it did nothing to prevent google from showing me all sorts of scary (sic) stuff when running a search on google images.  I guess this bit will just have to be covered by GOOD PARENTING! (Discussed in Step 2 - easier said than done...)
  4. Impliment openDNS or similar on border router (ADSL router).  Now nothing in the network will have access to said scary stuff except google image searches that is.
  5. Build small Linux desktop for child. -- Here it is.  The bit about Linux.  I intend to revive an old P100 or something slow like that I have lying around with a stable 12.04 (LTS) version of Ubuntu but with LXDE desktop to keep things sane.  Then I will configure PAM to restrict login times for the boy's user account, and then show some pop up warning (called by cron) so the boy has a chance to log out gracefully.  Because we don't want him on the thing all the time.  I have heard that kids will be up all night (just like me - it's now 00:25 in the morning) using their 'puters.

So here are some links that might be useful.

www.opendns.com
http://www.informit.com/articles/article.aspx?p=165226

So as I progress through all this, I will post here about how it all goes.

Of course, if I was LOADED,$$$, I would buy the boy a MAC.  Apparently they have this sort of functionality built in.

Saturday, February 18, 2012

OGG to MP3 Conversion with ID3 tags

I spend ages every time I need to do this to remember the correct format of the command:


here it is at last:


/usr/bin/find -type f -name "*.ogg" | while read file; do echo "working on $file"; /usr/bin/gst-launch filesrc location="${file}" ! oggdemux ! vorbisdec ! audioconvert ! lame quality=4 ! id3v2mux ! filesink location="`/usr/bin/dirname "$file"`/`/usr/bin/basename "$file" .ogg`.mp3" 2>&1 && /bin/rm "$file"; done | /usr/bin/tee -a $HOME/ogg2mp3results.txt


Find all the files
Convert them into mp3 with quality = 4 which is something like 44000khz and br=127kbit
Save them with an mp3 extension next to the original ogg
Remove the ogg file
Tee the output into a log.


HA!