Wednesday, August 31, 2011

Configuring MySQL on Redhat Linux while utilising network (iSCSI) storage.

Challenges:
MySQL only allows a single location to be specified as the default data store for all databases.

Consider the following typical scenario:
  • A database server running one instance of MySQL has its datastore located at /mysql/databases.
  • This server will contain multiple databases for multiple applications.
  • The database tables are a mixture of InnoDB and MYISAM storage engines.
  • The storage is provided by a SAN and separate storage volumes (iSCSI targets) are provisioned for each database.
In this scenario, you would want all your MYISAM files (*.MYI, *.MYD, *.frm) and all your InnoDB table spaces to be contained on the same iSCSI volume on a per-database basis.

Solution:

This is easily achieved by mounting the iSCSI target on a sub-folder of /mysql/databases. For example:

Mount iqn.openfiler.com:2006-06:blog-db-storage on /mysql/databases/blog

Here is a sequence of steps that need to be carried out to make this work.

  1. in /etc/my.cnf set datastore=/mysql/databases
  2. in /etc/my.cnf set innodb_file_per_table (http://dev.mysql.com/doc/refman/5.0/en/innodb-multiple-tablespaces.html)
  3. Configure SELINUX and user/group rights appropriately for MySQL. There is heaps on this on the internet but please comment if you would like me to add some detail here.
  4. Start the mysqld service
  5. configure the mysql root password if necessary
  6. login to mysql as root
  7. create the blog database (create database blog;) This creates the blog directory in /mysql/databases/blog At this point there is nothing inside it.
  8. log out of mysql.
  9. Mount your iscsi storage volume on /mysql/databases/blog
  10. Configure your system so that the iscsi storage volume has a persistence and is always mounted on this location. Redhat have a great tutorial on how to do this. Basically it involves checking the WID of your iscsi target and then applying that to a udev rule that creates a consistently named block device under /dev
  11. Now go ahead and create your tables. All the MYISAM and InnoDB tables will be created inside the /mysql/databases/blog directory.
Code / Scripts:

I am happy to provide scripts or code if people want. The following links should get you started.

Links:

Monday, August 29, 2011

MUST FOLLOW PROCESS...

MUST FOLLOW PROCESS.... MUST FOLLOW PROCESS...

Starting with the premise that we need our customers to be happy with us, what do we need to do as an organisation to keep them that way?

We are working through steps to improve our customer satisfaction at work by carefully considering how our daily work routines, processes and actions impact our customers.

Consider this fairly typical example of how an organisation might respond to a customer request:

[NOTE: This is a little bit (A LOT) satirical and not meant to in anyway be a real representation of an actual process. It's designed to highlight how strict adherence to process can lead to unhappy customers.]

A customer needs a new type of widget. Your organisation sells gadgets that are very similar to widgets so they approach you to design and build one for them. They are willing to pay for the service but they need this new widget to work for them.

Luckily your organisation has a process to handle this type of request. It's called Customer Request Acceptance Process. It is a sequence of actions scraped out of a managerial text book written in the early 1900's.

So the sales rep in your organisation finds the appropriate template and begins...

  1. The sales rep spends 2 hours working out how best to express his customer's requirements on the template. The template is designed to be all encompassing providing options for everything. Clearly (from looking at the template) anything not covered by it's fields are not valid for submission. [Templates have their place but be careful of over detailing them - they get confusing. A good place to start is a subject and details box. Leave the rest to the technical guys to nut out.]
  2. The working group that handle inbound requests receive the completed form and instantly find gaps in information. They make assumptions about some of the more obvious omissions and request clarification from the sales rep on the others. [Leave the mind reading to the professionals]
  3. The sales rep is very confident that he understands 90% of the questions and answers them straight away and on the other 10% he seeks clarification. [More mind reading...]
  4. Finally the requirements are gathered (from the sales rep and via a series of mini assumptions about unimportant details)
  5. A quote is hastily written up and the sales rep informed. The sales rep sells the solution to the customer. [the solution has to be sold??? I thought the customer asked for it in the first place. Be careful! If you have to **sell** the solution to a customer request then it's probably not what they wanted.]
  6. The tech team receive the go-ahead and begin work on the widget. The widget is FANTASTIC. It's so damn clever, some individuals are already talking about it to their contacts in an effort to drum up even more revenue. [Woooah there Silver, Does it work for the customer? Has anyone asked them?]
  7. The widget is delivered more or less on schedule. [Good work guys - go eat pizza and have a celebration.]
  8. The customer finds that it sort-of does what it's supposed to do but they don't complain because they have come to expect this sort of thing from the organisation.[What??? This is acceptable to you Mr. Customer? Find another provider already!]
  9. Your organisation sends the customer a survey asking how happy they are and soon it becomes apparent that they are not. [By sending a survey out you send a message saying that you need to know what's wrong before you can fix it. If you don't fix it your customer will assume your are being cynical about the whole survey thing and not bother answering it.]

My point with this extremely obvious example is that:

a happy customer is very simply a customer that you talk to all the time!

So while your organisation might have a process and tools to manage that process, it's useless if you don't know what your customer really wants.

Some customers are not too good at telling us what they really want.
Most of the time, I suggest that we are not placing the appropriate individuals in our organisations together. IE: Customer end user of WIDGET next to the provider technical person designing the WIDGET. It makes sense that these two should work closely together.

Providers should develop fast prototypes and be less precious about the their solutions in case they end up not being fit for purpose. Every time a mistake is corrected, your product is improved.

It is impossible to gather all the requirements in one go. Requirements have a way of evolving as the design takes shape.

No amount of technology based tools to manage your processes will help you if your processes do not include the customer at every step.

Every step in your process that does not include the customer results in your product diverging further from the customer's requirements.

We should all evaluate all the time how what we do impacts the customer. If you don't know how your job impacts your customers, you should make it your number one concern to find out. Customers pay your salary. Not your boss or your manager.

Thursday, July 21, 2011

REBLOG!!! Sticky bit, suid and guid

I have to reblog this article in it's full just in case the original author removes it.

Here is the original.
http://bashscript.blogspot.com/2010/03/unixlinux-advanced-file-permissions.html

Here in plain text to ** FORCE ** it into my rubbish layout.

UNIX/Linux Advanced File Permissions - SUID,SGID and Sticky Bit


After you have worked for a while with Linux you discover probably
that there is much more to file permissions than just the "rwx" bits.
When you look around in your file system you will see "s" and "t"

$ ls -ld /tmp
drwxrwxrwt 29 root root 36864 Mar 21 19:49 /tmp

$ which passwd
/usr/bin/passwd

$ ls -l /usr/bin/passwd
-rwsr-xr-x 1 root root 22984 Jan 6 2007 /usr/bin/passwd

What is this "s" and "t" bit? The vector of permission bits is really
4 * 3 bits long. Yes there are 12 permission bits,not just 9.The first
three bits are special and are frequently zero. And you almost always
learn about the trailing 9 bits first.Some people stop there and never
learn those first three bits.

The forth permission bit is used only when a special mode of a file
needs to be set. It has the value 4 for SUID, 2 for SGID and 1 for the
sticky bit. The other 3 bits have their usual significance.

Here we will discuss about the 3 special attributes other than the
common read/write/execute:

1.Set-User-Id (SUID)
2.Set-Group-Id (SGID)
3.Sticky Bit

Set-User_Id (SUID): Power for a Moment:

By default, when a user executes a file, the process which results in
this execution has the same permissions as those of the user. In fact,
the process inherits his default group and user identification.

If you set the SUID attribute on an executable file, the process res-
ulting in its execution doesn't use the user's identification but the
user identification of the file owner.

The SUID mechanism,invented by Dennis Ritchie,is a potential security
hazard. It lets a user acquire hidden powers by running such a file
owned by root.

$ ls -l /etc/passwd /etc/shadow /usr/bin/passwd
-rw-r--r-- 1 root root 2232 Mar 15 00:26 /etc/passwd
-r-------- 1 root root 1447 Mar 19 19:01 /etc/shadow

The listing shows that passwd is readable by all, but shadow is unre-
adable by group and others. When a user running the program belongs to
one of these two categories (probably, others), so access fails in the
read test on shadow. suppose normal user wants to change his password,
How can he do that? He can do that by running /usr/bin/passwd. Many
UNIX/Linux programs have a special permission mode that lets users
update sensitive system files –like /etc/shadow --something they can't
do directly with an editor. This is true of the passwd program.

$ ls -l /usr/bin/passwd
-rwsr-xr-x 1 root root 22984 Jan 6 2007 /usr/bin/passwd

The s letter in the user category of the permission field represents a
special mode known as the set-user-id (SUID). This mode lets a process
have the privileges of the owner of the file during the instance of
the program. Thus when a non privileged user executes passwd, the eff-
ective UID of the process is not the user's, but of root's – the owner
of the program. This SUID privilege is then used by passwd to edit
/etc/shadow.

What is effective user-id:

Every process really has two user IDs: the effective user ID and the
real user ID. (Of course, there's also an effective group ID and real
group ID.Just about everything that's true about user IDs is also true
about group IDs) Most of the time,the kernel checks only the effective
user ID. For example, if a process tries to open a file, the kernel
checks the effective user ID when deciding whether to let the process
access the file.

Save the following script under the name reids.pl and make it
executable (chmod 755 reids.pl).

#!/usr/bin/perl
# print real UID
print "Real UID: $<\n";
# print real GID
print "Real GID: $(\n";
# print effective UID
print "Effective UID: $>\n";
# print effective GID
print "Effective GID: $)\n";

check file permissions:

$ ls -l reids.pl
-rwxr-xr-x 1 venu venu 203 Mar 24 10:40 reids.pl

Note: For security reasons the s-bit works only when used on binaries
(compiled code) and not on scripts (an exception are perl scripts).
Scripts,i.e. programs that cannot be executed by the kernel directory
but need an interpreter such as the Bourne shell or Java,can have
their setuid bit set, but it doesn't have any effect. There are some
platforms that honor the s bits even on scripts ( some System V vari-
ants, for example), but most systems don't because it has proven such
a security headache - most interpreters simply aren't written with
much security in mind. Set the SUID bit on shell script is useless,
that's why I am using perl script here.

When you run the script you will see that the process that runs it
gets your user-ID and your group-ID:

$ ./reids.pl
Real UID: 500
Real GID: 500 500
Effective UID: 500
Effective GID: 500 500

Note: If you get an error like this:
Can't do setuid (cannot exec sperl)

In Debian install perl-suid using following command:
apt-get install perl-suid

In Centos install perl-suidperl using following command:
yum install perl-suidperl

Now change owner ship to another user (Do it as an administrator).

# chown king /home/venu/reids.pl
# ls -l /home/venu/reids.pl
-rwxr-xr-x 1 king venu 203 Mar 24 10:40 /home/venu/reids.pl

Now run the script again.

$ ./reids.pl
Real UID: 500
Real GID: 500 500
Effective UID: 500
Effective GID: 500 500

What you observed, the output of the program depends only on the user
that runs it and not the one who owns the file.

How to assign SUID permission:

The SUID for any file can be set (mostly by the superuser) with a
special syntax of the chmod command. This syntax uses the character s
as the permission. Now add SUID permission to the script reids.pl :

# chmod u+s /home/venu/reids.pl (Do it from root account)

Now return from the super user mode to the usual non privileged mode.

$ ls -l reids.pl
-rwsr-xr-x 1 king venu 203 Mar 24 10:40 reids.pl

To assign SUID in an absolute manner, simply prefix 4 to whatever
octal string you would otherwise use (like 4755 instead of 755).

The file reids.pl is owned by king and has the s-bit set where norma-
lly the x is for the owner of the file. This causes the file to be
executed under the user-ID of the user that owns the file rather than
the user that executes the file. If venu runs the program then this
looks as follows:

$ perl reids.pl
Real UID: 500
Real GID: 500 500
Effective UID: 503
Effective GID: 500 500

Effective user id of process is 503, this is not the venu's , but of
king's - the owner of the program. As you can see this is a very powe-
rful feature especially if root owns the file with s-bit set. Any user
can then do things that normally only root can do.

Caution: When you write a SUID program then you must make sure that
it can only be used for the purpose that you intended it to be used.
As administrator, you must keep track of all SUID programs owned by
root that a user may try to create or copy. The find command easily
locate them:

# find /home -perm -4000 -print | mail root

The extra octal bit (4) signifies the SUID mode, but find treats the
"–" before 4000 as representing any other permissions.

Set-Group_Id (SGID):

The set-group-id (SGID) is similar to SUID except that a program with
SGID set allows the user to have the same power as the group which
owns the program. The SGID bit is 2,and some typical examples could be
chmod g+s reids.pl or chmod 2755 reids.pl.
You can remove SGID bit using following commands:

$ chmod g-s reids.pl
$ chmod 755 reids.pl (Absolute manner)

It is really useful in case you have a real multi-user setup where
users access each others files. As a single homeuser I haven't really
found a lot of use for SGID. But the basic concept is the same as the
SUID,Similar to SUID, SGID also grants privileges and access rights to
the process running the command, but instead of receiving those of the
file's owner it receives those of the file's group. In other words,the
process group owner will be set to the file's group.

I explain it with an example. I have created two user accounts king
and venu with same home directory project. king belongs to king and
development groups, venu belongs to venu and development groups.

# groups king venu
king : king development
venu : venu development

venu's default group is venu and king's default group is king.

Login as king and create reids.pl file again and make it executable
(using chmod 755 reids.pl) .

$ id
uid=503(king) gid=503(king) groups=501(development),503(king)
$ ls -l reids.pl
-rwxr-xr-x 1 king development 203 Mar 25 19:00 reids.pl

Now login as venu and run the program:

$ id
uid=501(venu) gid=504(venu) groups=501(development),504(venu)
$ perl reids.pl
Real UID: 501
Real GID: 504 504 501
Effective UID: 501
Effective GID: 504 504 501

The effective GID of the process is the venu's,but not of the king's
-the owner of the program.

Now login as king and assign SGID bit to reids.pl program:

$ chmod 2755 reids.pl; ls -l reids.pl
-rwxr-sr-x 1 king development 203 Mar 25 19:00 reids.pl

Now login as venu and run the reids.pl program:

$ perl reids.pl
Real UID: 501
Real GID: 504 504 501
Effective UID: 501
Effective GID: 501 504 501

Real GID and Effective GID are different,here Effective GID is the
king's - the owner of the program.

Set SGID on a directory:

When SGID is set on a directory it has a special meaning. Files crea-
ted in a directory with SGID set will inherit the same group ownership
as the directory itself,not the group of the user who created the file.
If the SGID is not set the file's group ownership corresponds to the
user's default group.

In order to set the SGID on a directory or to remove it, use the
following commands:

$ chmod g+s directory or $ chmod 2755 directory
$ chmod g-s directory or $ chmod 755 directory

As I mentioned earlier venu and king's home directory is same that is
/home/project. I changed group ownership of /home/project directory
to development.

# ls -ld /home/project/
drwxrwxr-x 16 root development 4096 Mar 26 00:22 /home/project/

Now login as king and create a temp file.

$ whoami
king
$ pwd
/home/project/
$ touch temp; ls -l temp
-rw-r--r-- 1 king king 0 Mar 26 12:34 temp

You can see from the ls output that the group owner for project is
development, and that the SGID bit has not been set on the directory
yet. When king creates a file in project, the group for the file is
king (king's primary gid).

Set SGID bit on project directory. For that login as administrator
and set SGID bit using following command:

# chmod g+s /home/project/
# ls -ld /home/project/
drwxrwsr-x 15 root development 4096 Mar 26 12:34 /home/project/

From the ls output above, you know the SGID bit is set because of the
s in the third position of the group permission set,which replaces the
x in the group permissions.

Now login as king and create temp2 file.

$ whoami
king
$ touch temp2; ls -l temp2
-rw-r--r-- 1 king development 0 Mar 26 13:49 temp2

Notice the group ownership for temp2 file. It inherits group permiss-
ion from the parent directory.

Enabling SGID on a directory is extremely useful when you have a
group of users with different primary groups working on the same set
of files.

For system security reasons it is not a good idea to set many
program's set user or group ID bits any more than necessary,since this
can allow an unauthorized user privileges in sensitive system areas.If
the program has a flaw that allows the user to break out of the inten-
ded use of the program, then the system can be compromised.

Sticky bit:

The sticky bit(also called the saved text bit) is the last permission
bit remaining to be discussed. It applies to both regular files and
directories. When applied to a regular file, it ensures that the text
image of a program with the bit set is permanently kept in the swap
area so that it can be reloaded quickly when the program's turn to use
the CPU arrives. Previously, it made sense to have this bit set for
programs like vi and emacs. Today,machines with ultra-fast disk drives
and lots of cheap memory don't need this bit for ordinary files and
that is also useless.

However, the sticky bit become a useful security feature when used
with a directory. The UNIX/Linux system allows users to create files
in /tmp, but none can delete files not owned by him. That's possible
because sticky bit set for /tmp directory.

The /tmp directory is typically world-writable and looks like this
in a listing:

# ls -ld /tmp
drwxrwxrwt 32 root root 36864 Mar 27 12:38 /tmp

Everyone can read,write and access the directory.The t indicates that
only the user (root and owner of the directory,of course) that created
a file in this directory can delete that file.

In order to set or to remove the sticky bit, use the following
commands:

$ chmod +t directory or $ chmod 1754 directory
$ chmod -t directory or $ chmod 754 directory

Note: 754 permissions for a directory are powerful enough to guard
your directories from intruders with malicious intentions, that's why
I used 754 as default,if yow want you can change it.

Example:

I logged in as king and created a temp file.

$ whoami
king
$ pwd
/home/project/
$ touch temp; ls -l
-rw-r--r-- 1 king king 0 Mar 27 13:44 temp

Now logged in as venu and try to delete temp file.

$ whoami
venu
$ rm temp
rm: remove write-protected regular empty file `temp'? Y
$ ls temp
ls: temp: No such file or directory

So what happened? venu deleted file owned by king.

Assign sticky bit to the project directory.As a owner of the directory
or administrator.

# chmod +t /home/project
# ls -ld /home/project/
drwxrwxr-t 15 root development 4096 Mar 27 13:46 /home/project/

From the ls output above, you know the sticky bit is set because of
the t in the third position of the other permission set,which replaces
the x in the other permissions.

Now repeat same steps again,then you get the following message:

$ whoami
venu
$ ls -l temp
-rw-r--r-- 1 king king 0 Mar 27 17:36 temp
$ rm temp
rm: remove write-protected regular empty file `temp'? y
rm: cannot remove `temp': Operation not permitted


Observation: Login as normal user and create a file.
[venu@localhost ~]$ touch sample
[venu@localhost ~]$ ls -l sample
-rw-rw-r-- 1 venu venu 0 Dec 21 03:41 sample

Now change permissions to 644

[venu@localhost ~]$ chmod 644 sample
[venu@localhost ~]$ ls -l sample
-rw-r--r-- 1 venu venu 0 Dec 21 03:41 sample

Now assign SUID permission.

[venu@localhost ~]$ chmod u+s sample
[venu@localhost ~]$ ls -l sample
-rwSr--r-- 1 venu venu 0 Dec 21 03:41 sample

After setting SUID, if you see 'S' then it means that the file has no
executable permissions for that user.

Now remove SUID permission and change permissions to 744. Then assign
SUID permission. You should see a smaller 's' in the executable permi-
ssion position.

[venu@localhost ~]$ chmod u-s sample
[venu@localhost ~]$ chmod 744 sample
[venu@localhost ~]$ chmod u+s sample
[venu@localhost ~]$ ls -l sample
-rwsr--r-- 1 venu venu 0 Dec 21 03:41 sample

Same is applicable for SGID and Stickybit.


Posted by venu k at 10:51 AM
Labels: Article

Tuesday, March 1, 2011

Switch Google Desktop Search to search desktop by default

I recently installed the latest version of Google Desktop. I found, however, that when searching either from the sidebar searchbox or the Ctrl Ctrl popup box, GD would always select Search Web by default.

This can now be changed using a small registry edit on Windows XP it is:
HKEY_CURRENT_USER\Software\Google\Google Desktop\Deskbar\ change search_type to "Search Desktop"

That's it. Hopefully this will save some of you some time.

Wednesday, January 5, 2011

Some programming guidlines

This is a very simple post. I found these principals very useful in solving a problem I have with a particular project I am currently working on...

http://en.wikipedia.org/wiki/Unix_philosophy

1.Small is beautiful.
2.Make each program do one thing well.
3.Build a prototype as soon as possible.
4.Choose portability over efficiency.
5.Store data in flat text files.
6.Use software leverage to your advantage.
7.Use shell scripts to increase leverage and portability.
8.Avoid captive user interfaces.
9.Make every program a filter.


Have a read. It's well worth remembering these guidelines. Even if you have read them before, it might be useful to refresh your memmory.