In my view, a best practice can only be a best practice if it 1) can be justified as to why it's advantageous and 2) can be flexible to adapt to multiple situations. The following five essential best practices were derived from my years of experience working with Oracle systems large and small.
#1: Multiple Oracle Homes
My favorite best practice is the one about multiple Oracle Homes. Here it how it goes. When applying a patch or a patchset, I recommend against applying to the existing Oracle Home. Instead, I suggest creating a new Oracle Home, and apply the patches there.
I create the first Oracle Home at /app/oracle/db_1, for instance. When a patch comes out, I install the whole Oracle software in a different home -- /app/oracle/db_2 -- and then apply the patch there. During the process of installation and patch application, the database is up and running, as it runs off the home /db_1. When the outage window comes, all I have to do is to shut down Oracle; change Oracle Home to db_2 and bring the database up. If there is a problem, I can reset the Oracle Home back to the old one.
So, here is the conventional approach:
Shut down the database
Apply patch to the Oracle Home
Start the database
In case of problems:
Shut down the database
Roll back the patch
Start the database Steps 2 and 6 could take as much as three hours depending on the amount of patching. The database is down during these times.
In the new approach:
Install new Oracle Home
Apply the patch to the new Home
Shut down the database
Change Oracle Home to the new location
Start the database
In case of problems:
Shut down the database
Change Oracle Home to the old one
Start the database The database is down only during steps 4 and 8, which takes a couple of minutes at the most, not hours.
So, here are the advantages:
The downtime is significantly reduced, to one 60th of the original time.
The risk is reduced significantly, as there is no rolling back a patch; you just go to the older version.
You can perform a "diff" on these two homes to see what changed. You can see the differences across multiple homes as well.
You can take several databases running on the same server to the new Oracle Home one by one.
You can see the various Oracle Homes and what patch level they are on using the inventory. The only negative part is the space consumption -- you need space for two Oracle Homes. But considering a typical Oracle Home takes about 4 GB or less, this aspect of the suggestion is trivial.
#2: Set audit trail to DB
Set the audit trail to DB by placing the parameter AUDIT_TRAIL = DB in the initialization parameter file during the database creation. Setting this parameter does not start the auditing, because an explicit AUDIT command must be given on the object. But the parameter must be set to a value other than FALSE (the default) for the command to take effect. Being a non-dynamic parameter, the database must be bounced to change the value of AUDIT_TRAIL. To save the trouble and avoid an outage, always set the value to DB, even if you never intend to audit anything. It does not break anything and you will always be ready to audit when the time comes.
#3: Don't use .log
Don't use .log as the extension of redo logs. Someone may run a script to remove all the log files assuming they are redundant and you will end up losing the online redo logs as well, forcing a database recovery. Instead, name them with extension "redo" or "rdo."
#4: Preview RMAN Restore
Preview RMAN Restore to identify all the various backup pieces that will be used in the recovery process without doing an actual recovery. This eliminates any surprises from missing pieces during an actual recovery process.
#5: Create a new Oracle user for clients running on the same server as the DB
The Oracle Database server software also contains the client piece, which allows the clients to connect to the database on the same server. But as a best practice do not use the same user or the software; use a new one. For instance, if "oracle" is the user to install Oracle software, create a new user called, say, "oraapp" and install the client-only software using that user. The user "oraapp" should not be part of the dba or the oinstall group; so this user can't log on to the database as sysdba. Create a new group called "appgrp" and assign the user oraaap to this group. All the application users on the box should also be part of the appgrp group. This way they can use the sqlplus, sqlldr and other executables on the server, but be able to connect as sysdba.
The common practice is to use the client software in the same user as the database software owner; but starting with 10.2, Oracle has changed the security policy that takes away the global execution permissions from the Oracle Home. So the only option is to let app users be part of the dba group or change the permissions on Oracle Home -- both make the database extremely vulnerable.
Friday, August 24, 2007
Securing Oracle databases
In order to properly secure your Oracle databases, the following steps are necessary. I tested these on Oracle 8.1.6 and 8.1.7.
Check profiles
Check the DB user
select * from all_users;
Check the system privilege for PUBLIC grants
select * from dba_sys_privs
where GRANTEE = 'PUBLIC'
or privilege like '%ANY%';
Check Profiles available on the server
select * from dba_profiles;
Decide your profile method and create new profile if necesary
CREATE PROFILE "" LIMIT
CPU_PER_SESSION DEFAULT
CPU_PER_CALL DEFAULT
CONNECT_TIME DEFAULT
IDLE_TIME DEFAULT
SESSIONS_PER_USER DEFAULT
LOGICAL_READS_PER_SESSION DEFAULT
LOGICAL_READS_PER_CALL DEFAULT
PRIVATE_SGA DEFAULT
COMPOSITE_LIMIT DEFAULT
FAILED_LOGIN_ATTEMPTS DEFAULT
PASSWORD_LOCK_TIME DEFAULT
PASSWORD_GRACE_TIME DEFAULT
PASSWORD_LIFE_TIME DEFAULT
PASSWORD_REUSE_MAX DEFAULT
PASSWORD_REUSE_TIME DEFAULT
PASSWORD_VERIFY_FUNCTION DEFAULT;
LIMIT Explain Table
===================
CPU_PER_SESSION Total amount of CPU time allowed in a
session. The limit is expressed in seconds.
CPU_PER_CALL Total amount of CPU time allowed in a
call (a parse, execute, or fetch).
The limit is expressed in seconds.
CONNECT_TIME Maximum elapsed time allowed for a
session. The limit is expressed in minutes.
IDLE_TIME Maximum idle time allowed in a session.
Idle time is a continuous period of inactive
time during a session. Long-running queries and
other operations are not subject to this limit. The
limit is expressed in minutes.
SESSIONS_PER_USER Maximum number of concurrent
sessions allowed for a user.
LOGICAL_READS_PER_SESSION Total number of data block reads
allowed in a session. The limit includes
blocks read from memory and disk.
LOGICAL_READS_PER_CALL Maximum number of data block reads
allowed for a call (a parse, execute,
or fetch) to process a SQL statement.
PRIVATE_SGA Maximum amount of private space a
session can allocate in the shared
pool of the System Global Area (SGA). The
Private SGA limit applies only if
you are using the multi-threaded
server architecture. The limit is expressed in
kilobytes (Kbytes).
COMPOSITE_LIMIT Total resource cost for a session.
The resource cost for a session is the
weighted sum of the CPU time used in the session,
the connect time, the number of reads made in the
session, and the amount of private SGA space
allocated.
FAILED_LOGIN_ATTEMPTS Limits the number of failed logon attempts
allowed before a user is locked out
from the account.
PASSWORD_LOCK_TIME Specifies the number of days the account is
locked after failing the specified
number of logon attempts. If UNLIMITED is specified,
only the database administrator can unlock the
account.
PASSWORD_GRACE_TIME Limits the number of days during which a
password can be changed following the first
successful login after password expiration.
PASSWORD_LIFE_TIME Limits the number of days after which a
password expires.
PASSWORD_REUSE_MAX Specifies the number of times a password must
be changed before it can be reused.
PASSWORD_REUSE_TIME Limits the number of days before a password
can be reused after it expires.
PASSWORD_VERIFY_FUNCTION Allows a PL/SQL routine to be used for password
verification when users who are assigned this profile
log into a database. This PL/SQL routine must be
locally available for execution on the database to
which this profile applies. Oracle provides a default
script (utlpwdmg.sql), however, you can also create
your own routine, or use third-party software as an
alternative. The password verification routine must
be owned by SYS.
NULL No password verification is set by default.
Change DEFAULT users' passwords
Change DEFAULT passwords. Login as sys user
alter useridentified by ;
Set/Change LISTENER password
Run LSNRCTL utility from command line
lsnrctl
Result:
LSNRCTL for 32-bit Windows: Version 8.1.6.0.0 - Production on 06-JAN-2002 19:54:18
(c) Copyright 1998, 1999, Oracle Corporation. All rights reserved.
Welcome to LSNRCTL, type "help" for information.
LSNRCTL>
Enter change_password - Result:
Old password: (Enter the old password and press ENTER. If there isn't one, press ENTER)
New password: (Enter a new password)
Reenter new password: (Reenter the password)
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=)(PORT=1521)))
Password changed for LISTENER
The command completed successfully
LSNRCTL>
Enter: set password - To activate the password. Result:
Password: (Enter the password and press ENTER)
The command completed successfully
LSNRCTL>
Enter: save_config - To save the LISTENER configuration. Result:
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=)(PORT=1521)))
Saved LISTENER configuration parameters.
Listener parameter file - D:OracleOra8i_816
etworkdminlistener.ora
Old parameter file - D:OracleOra8i_816
etworkdminlistener.bak
The command completed successfully
LSNRCTL>
To START/STOP the LISTENER:
-- lsnrctl
-- set password
-- Enter the password
-- start/stop
Check profiles
Check the DB user
select * from all_users;
Check the system privilege for PUBLIC grants
select * from dba_sys_privs
where GRANTEE = 'PUBLIC'
or privilege like '%ANY%';
Check Profiles available on the server
select * from dba_profiles;
Decide your profile method and create new profile if necesary
CREATE PROFILE "
CPU_PER_SESSION DEFAULT
CPU_PER_CALL DEFAULT
CONNECT_TIME DEFAULT
IDLE_TIME DEFAULT
SESSIONS_PER_USER DEFAULT
LOGICAL_READS_PER_SESSION DEFAULT
LOGICAL_READS_PER_CALL DEFAULT
PRIVATE_SGA DEFAULT
COMPOSITE_LIMIT DEFAULT
FAILED_LOGIN_ATTEMPTS DEFAULT
PASSWORD_LOCK_TIME DEFAULT
PASSWORD_GRACE_TIME DEFAULT
PASSWORD_LIFE_TIME DEFAULT
PASSWORD_REUSE_MAX DEFAULT
PASSWORD_REUSE_TIME DEFAULT
PASSWORD_VERIFY_FUNCTION DEFAULT;
LIMIT Explain Table
===================
CPU_PER_SESSION Total amount of CPU time allowed in a
session. The limit is expressed in seconds.
CPU_PER_CALL Total amount of CPU time allowed in a
call (a parse, execute, or fetch).
The limit is expressed in seconds.
CONNECT_TIME Maximum elapsed time allowed for a
session. The limit is expressed in minutes.
IDLE_TIME Maximum idle time allowed in a session.
Idle time is a continuous period of inactive
time during a session. Long-running queries and
other operations are not subject to this limit. The
limit is expressed in minutes.
SESSIONS_PER_USER Maximum number of concurrent
sessions allowed for a user.
LOGICAL_READS_PER_SESSION Total number of data block reads
allowed in a session. The limit includes
blocks read from memory and disk.
LOGICAL_READS_PER_CALL Maximum number of data block reads
allowed for a call (a parse, execute,
or fetch) to process a SQL statement.
PRIVATE_SGA Maximum amount of private space a
session can allocate in the shared
pool of the System Global Area (SGA). The
Private SGA limit applies only if
you are using the multi-threaded
server architecture. The limit is expressed in
kilobytes (Kbytes).
COMPOSITE_LIMIT Total resource cost for a session.
The resource cost for a session is the
weighted sum of the CPU time used in the session,
the connect time, the number of reads made in the
session, and the amount of private SGA space
allocated.
FAILED_LOGIN_ATTEMPTS Limits the number of failed logon attempts
allowed before a user is locked out
from the account.
PASSWORD_LOCK_TIME Specifies the number of days the account is
locked after failing the specified
number of logon attempts. If UNLIMITED is specified,
only the database administrator can unlock the
account.
PASSWORD_GRACE_TIME Limits the number of days during which a
password can be changed following the first
successful login after password expiration.
PASSWORD_LIFE_TIME Limits the number of days after which a
password expires.
PASSWORD_REUSE_MAX Specifies the number of times a password must
be changed before it can be reused.
PASSWORD_REUSE_TIME Limits the number of days before a password
can be reused after it expires.
PASSWORD_VERIFY_FUNCTION Allows a PL/SQL routine to be used for password
verification when users who are assigned this profile
log into a database. This PL/SQL routine must be
locally available for execution on the database to
which this profile applies. Oracle provides a default
script (utlpwdmg.sql), however, you can also create
your own routine, or use third-party software as an
alternative. The password verification routine must
be owned by SYS.
NULL No password verification is set by default.
Change DEFAULT users' passwords
Change DEFAULT passwords. Login as sys user
alter user
Set/Change LISTENER password
Run LSNRCTL utility from command line
lsnrctl
Result:
LSNRCTL for 32-bit Windows: Version 8.1.6.0.0 - Production on 06-JAN-2002 19:54:18
(c) Copyright 1998, 1999, Oracle Corporation. All rights reserved.
Welcome to LSNRCTL, type "help" for information.
LSNRCTL>
Enter change_password - Result:
Old password: (Enter the old password and press ENTER. If there isn't one, press ENTER)
New password: (Enter a new password)
Reenter new password: (Reenter the password)
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=
Password changed for LISTENER
The command completed successfully
LSNRCTL>
Enter: set password - To activate the password. Result:
Password: (Enter the password and press ENTER)
The command completed successfully
LSNRCTL>
Enter: save_config - To save the LISTENER configuration. Result:
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=
Saved LISTENER configuration parameters.
Listener parameter file - D:OracleOra8i_816
etworkdminlistener.ora
Old parameter file - D:OracleOra8i_816
etworkdminlistener.bak
The command completed successfully
LSNRCTL>
To START/STOP the LISTENER:
-- lsnrctl
-- set password
-- Enter the password
-- start/stop
Wednesday, August 22, 2007
Variables in Shell
To process our data/information, data must be kept in computers RAM memory. RAM memory is divided into small locations, and each location had unique number called memory location/address, which is used to hold our data. Programmer can give a unique name to this memory location/address called memory variable or variable (Its a named storage location that may take different values, but only one at a time).
In Linux (Shell), there are two types of variable:
(1) System variables - Created and maintained by Linux itself. This type of variable defined in CAPITAL LETTERS.
(2) User defined variables (UDV) - Created and maintained by user. This type of variable defined in lower letters.
You can see system variables by giving command like $ set, some of the important System variables are:
style='width:99.0%;mso-cellspacing:.7pt;mso-yfti-tbllook:1184'>
In Linux (Shell), there are two types of variable:
(1) System variables - Created and maintained by Linux itself. This type of variable defined in CAPITAL LETTERS.
(2) User defined variables (UDV) - Created and maintained by user. This type of variable defined in lower letters.
You can see system variables by giving command like $ set, some of the important System variables are:
System
Variable
Meaning
BASH=/bin/bash
Our shell name
BASH_VERSION=1.14.7(1)
Our shell version name
COLUMNS=80
No. of columns for our screen
HOME=/home/vivek
Our home directory
LINES=25
No. of columns for our screen
LOGNAME=students
students Our logging name
OSTYPE=Linux
Our Os type
PATH=/usr/bin:/
Our path settings
PS1=[\u@\h \W]\$
Our prompt settings
PWD=/home/students/Common
Our current working directory
SHELL=/bin/bash
Our shell name
USERNAME=vivek
User name who is currently login
to this PC
NOTE
"Times New Roman"'> that Some of the above settings
can be different in your PC/Linux environment. You can print any of the above
variables contains as follows:
$ echo $USERNAME
$ echo $HOME
"Times New Roman"'>
Exercise:
1) If you want to print your home directory location then you give command:
a)$ echo $HOME
"Times New Roman"'>
OR
(b)$ echo HOME
"Times New Roman"'>
Caution:
"Times New Roman"'> Do not modify System variable this
can some time create problems.
How to write shell script
How to write shell script
Following steps are required to write shell script:
(1) Use any editor like vi or mcedit to write shell script.
(2) After writing shell script set execute permission for your script as follows
syntax:
chmod permission your-script-name
Examples:
$ chmod +x your-script-name
$ chmod 755 your-script-name
Note: This will set read write execute(7) permission for owner, for group and other permission is read and execute only(5).
(3) Execute your script as
syntax:
bash your-script-name
sh your-script-name
./your-script-name
Examples:
$ bash bar
$ sh bar
$ ./bar
NOTE In the last syntax ./ means current directory, But only . (dot) means execute given command file in current shell without starting the new copy of shell, The syntax for . (dot) command is as follows
Syntax:
. command-name
Example:
$ . foo
Now you are ready to write first shell script that will print "Knowledge is Power" on screen. See the common vi command list , if you are new to vi.
$ vi first
#
# My first shell script
#
clear
echo "Knowledge is Power"
After saving the above script, you can run the script as follows:
$ ./first
This will not run script since we have not set execute permission for our script first; to do this type command
$ chmod 755 first
$ ./first
First screen will be clear, then Knowledge is Power is printed on screen.
How Shell Locates the file (My own bin directory to execute script)
Tip: For shell script file try to give file extension such as .sh, which can be easily identified by you as shell script.
Exercise:
1)Write following shell script, save it, execute it and note down the it's output.
$ vi ginfo
#
#
# Script to print user information who currently login , current date & time
#
clear
echo "Hello $USER"
echo "Today is \c ";date
echo "Number of user login : \c" ; who | wc -l
echo "Calendar"
cal
exit 0
Following steps are required to write shell script:
(1) Use any editor like vi or mcedit to write shell script.
(2) After writing shell script set execute permission for your script as follows
syntax:
chmod permission your-script-name
Examples:
$ chmod +x your-script-name
$ chmod 755 your-script-name
Note: This will set read write execute(7) permission for owner, for group and other permission is read and execute only(5).
(3) Execute your script as
syntax:
bash your-script-name
sh your-script-name
./your-script-name
Examples:
$ bash bar
$ sh bar
$ ./bar
NOTE In the last syntax ./ means current directory, But only . (dot) means execute given command file in current shell without starting the new copy of shell, The syntax for . (dot) command is as follows
Syntax:
. command-name
Example:
$ . foo
Now you are ready to write first shell script that will print "Knowledge is Power" on screen. See the common vi command list , if you are new to vi.
$ vi first
#
# My first shell script
#
clear
echo "Knowledge is Power"
After saving the above script, you can run the script as follows:
$ ./first
This will not run script since we have not set execute permission for our script first; to do this type command
$ chmod 755 first
$ ./first
First screen will be clear, then Knowledge is Power is printed on screen.
| Script Command(s) | Meaning |
| $ vi first | Start vi editor |
| # # My first shell script # | # followed by any text is considered as comment. Comment gives more information about script, logical explanation about shell script. Syntax: # comment-text |
| clear | clear the screen |
| echo "Knowledge is Power" | To print message or value of variables on screen, we use echo command, general form of echo command is as follows syntax: echo "Message" |
How Shell Locates the file (My own bin directory to execute script)
Tip: For shell script file try to give file extension such as .sh, which can be easily identified by you as shell script.
Exercise:
1)Write following shell script, save it, execute it and note down the it's output.
$ vi ginfo
#
#
# Script to print user information who currently login , current date & time
#
clear
echo "Hello $USER"
echo "Today is \c ";date
echo "Number of user login : \c" ; who | wc -l
echo "Calendar"
cal
exit 0
What is Shell Script ?
Normally shells are interactive. It means shell accept command from you (via keyboard) and execute them. But if you use command one by one (sequence of 'n' number of commands) , the you can store this sequence of command to text file and tell the shell to execute this text file instead of entering the commands. This is know as shell script.
Shell script defined as:
"Shell Script is series of command written in plain text file. Shell script is just like batch file is MS-DOS but have more power than the MS-DOS batch file."
Why to Write Shell Script ?
Shell script can take input from user, file and output them on screen.
Useful to create our own commands.
Save lots of time.
To automate some task of day today life.
System Administration part can be also automated.
Shell script defined as:
"Shell Script is series of command written in plain text file. Shell script is just like batch file is MS-DOS but have more power than the MS-DOS batch file."
Why to Write Shell Script ?
Shell script can take input from user, file and output them on screen.
Useful to create our own commands.
Save lots of time.
To automate some task of day today life.
System Administration part can be also automated.
Tuesday, August 21, 2007
What is Linux Shell ?
What is Linux Shell ?
Computer understand the language of 0's and 1's called binary language.
In early days of computing, instruction are provided using binary language, which is difficult for all of us, to read and write. So in Os there is special program called Shell. Shell accepts your instruction or commands in English (mostly) and if its a valid command, it is pass to kernel.
Shell is a user program or it's environment provided for user interaction. Shell is an command language interpreter that executes commands read from the standard input device (keyboard) or from a file.
Shell is not part of system kernel, but uses the system kernel to execute programs, create files etc.
Several shell available with Linux including:
Shell Name Developed by Where Remark
BASH ( Bourne-Again SHell ) Brian Fox and Chet Ramey
Free Software Foundation
Most common shell in Linux. It's Freeware shell.
CSH (C SHell) Bill Joy University of California (For BSD) The C shell's syntax and usage are very similar to
the C programming language.
KSH (Korn SHell) David Korn AT & T Bell Labs --
TCSH
Tip: To find all available shells in your system type following command:
$ cat /etc/shells
Note that each shell does the same job, but each understand a different command syntax and provides different built-in functions.
In MS-DOS, Shell name is COMMAND.COM which is also used for same purpose, but it's not as powerful as our Linux Shells are!
Any of the above shell reads command from user (via Keyboard or Mouse) and tells Linux Os what users want. If we are giving commands from keyboard it is called command line interface ( Usually in-front of $ prompt, This prompt is depend upon your shell and Environment that you set or by your System Administrator, therefore you may get different prompt ).
Tip: To find your current shell type following command
$ echo $SHELL
Computer understand the language of 0's and 1's called binary language.
In early days of computing, instruction are provided using binary language, which is difficult for all of us, to read and write. So in Os there is special program called Shell. Shell accepts your instruction or commands in English (mostly) and if its a valid command, it is pass to kernel.
Shell is a user program or it's environment provided for user interaction. Shell is an command language interpreter that executes commands read from the standard input device (keyboard) or from a file.
Shell is not part of system kernel, but uses the system kernel to execute programs, create files etc.
Several shell available with Linux including:
Shell Name Developed by Where Remark
BASH ( Bourne-Again SHell ) Brian Fox and Chet Ramey
Free Software Foundation
Most common shell in Linux. It's Freeware shell.
CSH (C SHell) Bill Joy University of California (For BSD) The C shell's syntax and usage are very similar to
the C programming language.
KSH (Korn SHell) David Korn AT & T Bell Labs --
TCSH
Tip: To find all available shells in your system type following command:
$ cat /etc/shells
Note that each shell does the same job, but each understand a different command syntax and provides different built-in functions.
In MS-DOS, Shell name is COMMAND.COM which is also used for same purpose, but it's not as powerful as our Linux Shells are!
Any of the above shell reads command from user (via Keyboard or Mouse) and tells Linux Os what users want. If we are giving commands from keyboard it is called command line interface ( Usually in-front of $ prompt, This prompt is depend upon your shell and Environment that you set or by your System Administrator, therefore you may get different prompt ).
Tip: To find your current shell type following command
$ echo $SHELL
What Kernel Is?
What Kernel Is?
Kernel is hart of Linux Os.
It manages resource of Linux Os. Resources means facilities available in Linux. For e.g. Facility to store data, print data on printer, memory, file management etc .
Kernel decides who will use this resource, for how long and when. It runs your programs (or set up to execute binary files).
The kernel acts as an intermediary between the computer hardware and various programs/application/shell.
It's Memory resident portion of Linux. It performance following task :-
I/O management
Process management
Device management
File management
Memory management
Kernel is hart of Linux Os.
It manages resource of Linux Os. Resources means facilities available in Linux. For e.g. Facility to store data, print data on printer, memory, file management etc .
Kernel decides who will use this resource, for how long and when. It runs your programs (or set up to execute binary files).
The kernel acts as an intermediary between the computer hardware and various programs/application/shell.
It's Memory resident portion of Linux. It performance following task :-
I/O management
Process management
Device management
File management
Memory management
Where I can use Linux?
Where I can use Linux?
You can use Linux as Server Os or as stand alone Os on your PC. (But it is best suited for Server.) As a server Os it provides different services/network resources to client. Server Os must be:
Stable
Robust
Secure
High Performance
Linux offers all of the above characteristics plus its Open Source and Free OS. So Linux can be used as:
(1) On stand alone workstation/PC for word processing, graphics, software development, internet, e-mail, chatting, small personal database management system etc.
(2) In network environment as:
(A) File and Print or Application Server
Share the data, Connect the expensive device like printer and share it, e-mail within the LAN/intranet etc are some of the application.
(B) Linux sever cab be connected to Internet, So that PC's on intranet can share the internet/e-mail etc. You can put your web sever that run your web site or transmit the information on the internet.
So you can use Linux for:
Personal Work
Web Server
Software Development Workstation
Workgroup Server
In Data Center for various server activities such as FTP, Telnet, SSH, Web, Mail, Proxy, Proxy Cache Appliance etc
You can use Linux as Server Os or as stand alone Os on your PC. (But it is best suited for Server.) As a server Os it provides different services/network resources to client. Server Os must be:
Stable
Robust
Secure
High Performance
Linux offers all of the above characteristics plus its Open Source and Free OS. So Linux can be used as:
(1) On stand alone workstation/PC for word processing, graphics, software development, internet, e-mail, chatting, small personal database management system etc.
(2) In network environment as:
(A) File and Print or Application Server
Share the data, Connect the expensive device like printer and share it, e-mail within the LAN/intranet etc are some of the application.
(B) Linux sever cab be connected to Internet, So that PC's on intranet can share the internet/e-mail etc. You can put your web sever that run your web site or transmit the information on the internet.
So you can use Linux for:
Personal Work
Web Server
Software Development Workstation
Workgroup Server
In Data Center for various server activities such as FTP, Telnet, SSH, Web, Mail, Proxy, Proxy Cache Appliance etc
Who developed the Linux?
Who developed the Linux?
In 1991, Linus Torvalds studding Unix at the University, where he used special educational experimental purpose operating system called Minix (small version of Unix and used in Academic environment). But Minix had it's own limitations. Linus felt he could do better than the Minix. So he developed his own version of Minix, which is now know as Linux. Linux is Open Source From the start of the day. For more information on Linus Torvalds, please visit his home page.
In 1991, Linus Torvalds studding Unix at the University, where he used special educational experimental purpose operating system called Minix (small version of Unix and used in Academic environment). But Minix had it's own limitations. Linus felt he could do better than the Minix. So he developed his own version of Minix, which is now know as Linux. Linux is Open Source From the start of the day. For more information on Linus Torvalds, please visit his home page.
Saturday, August 18, 2007
Using rsync to backup data from one server to another over SSH.
Rsync is a great tool which can be used to do many tasks which involved copying/moving data. If privacy/security is of concern, which it always should be, you can use rsync to do all the copying/moving of data over SSH. Read through ”man rsync” to get deeper understanding of rsync. Here is my attempt to a short tutorial on rsync. Let us start with most simple example of using rsync over ssh.
rsync -ae ssh server1:/home /home/backups/server1_home_backup/
This command will download all the files/directories from /home on server1 and copies them to /home/backups/server1_home_backup/
-a = archive mode. This will preserve permissions, timestamps, etc.
-e = specify which remote shell to use. In our case, we want to use ssh which follow right after “e”
Let us improve on this and add couple more parameters:
rsync -zave ssh --progress server1:/home /home/backups/server1_home_backup/
-z = adds zip compression.
-v = verbose
–progress = my favorite parameter when I am doing rsync manually, not so good when you have it in cron. This show progress (how_many_files_left/how_many_files_total) and speed along with some other useful data.
Great.. we are moving along pretty good. Let us add some security to make sure things work the way we want to.
rsync --delete-after -zave ssh --progress server1:/home /home/backups/server1_home_backup/
–delete-after = this will delete files on backup server which are missing from source after ALL syncing is done. If you don’t care of having extra files on your backup server and have plenty of disk space to spare, do not use this parameter.
Lastly, one of the VERY handy parameters,
rsync --delete-after -zave ssh --progress server1:/home /home/backups/server1_home_backup/ -n
The -n (or –dry-run) parameter is great to use for testing. It will not transfer or delete any files, rather will report to you what it would have done if it was ran with out -n parameter. This way you can test it with out destroying or transfering data just to find out that is not what you wanted.
For further reading: man rsync
rsync -ae ssh server1:/home /home/backups/server1_home_backup/
This command will download all the files/directories from /home on server1 and copies them to /home/backups/server1_home_backup/
-a = archive mode. This will preserve permissions, timestamps, etc.
-e = specify which remote shell to use. In our case, we want to use ssh which follow right after “e”
Let us improve on this and add couple more parameters:
rsync -zave ssh --progress server1:/home /home/backups/server1_home_backup/
-z = adds zip compression.
-v = verbose
–progress = my favorite parameter when I am doing rsync manually, not so good when you have it in cron. This show progress (how_many_files_left/how_many_files_total) and speed along with some other useful data.
Great.. we are moving along pretty good. Let us add some security to make sure things work the way we want to.
rsync --delete-after -zave ssh --progress server1:/home /home/backups/server1_home_backup/
–delete-after = this will delete files on backup server which are missing from source after ALL syncing is done. If you don’t care of having extra files on your backup server and have plenty of disk space to spare, do not use this parameter.
Lastly, one of the VERY handy parameters,
rsync --delete-after -zave ssh --progress server1:/home /home/backups/server1_home_backup/ -n
The -n (or –dry-run) parameter is great to use for testing. It will not transfer or delete any files, rather will report to you what it would have done if it was ran with out -n parameter. This way you can test it with out destroying or transfering data just to find out that is not what you wanted.
For further reading: man rsync
How do you install and set up Samba in linux? [Redhat Enterprise(RHEL), CentOS, Fedora]
Setting up Samba “can” be complicated at times. Especially if you are looking for instructions online where there are WAY too many tutorials which go in to details about what configuration does what, etc. Well, this post is nothing like that. Here I just show you quick and easy way to install Samba, configure it, and set up the drive letter on your XP/Vista. NOTE: for using Samba with Vista, please see my previous post in which I talk about changing settings in Vista so you can connect to your Samba share: Windows Vista Installation
NOTE: This set up is very “open” and should not be used on servers which are facing the world. This is for private network where you trust all the computers and its users.
Installing Samba (using yum on CentOS and Fedora): yum install samba
Installing Samba (using rpm):
Obtain Samba rpm from rhn.redhat.com
rpm -ivh samba*.rpm
Configuring Samba:
cd /etc/samba
mv smb.conf smb.conf.backup
vi smb.conf
[global]
workgroup = wrkgrp
netbios name = smbserver
security = SHARE
load printers = No
default service = global
path = /home
available = No
encrypt passwords = yes
[share]
writeable = yes
admin users = smbuser
path = /home/share
force user = root
valid users = smbuser
public = yes
available = yes
save and exit
adduser smbuser #add unix account
passwd smbuser #set unix account password
smbpasswd -a smbuser #lets create same user account on samba
/etc/init.d/smb restart
Now let us setup drive letter on our Windows so we can easily access these files.
Start -> run -> cmd
At the prompt type: net use z: \\ip_of_your_samba_server\share /user: smbuser password_you_assigned
That is it! At this point you have successfully set up Samba under Linux and are now successfully connected to your share from your Windows machine
NOTE: This set up is very “open” and should not be used on servers which are facing the world. This is for private network where you trust all the computers and its users.
Installing Samba (using yum on CentOS and Fedora): yum install samba
Installing Samba (using rpm):
Obtain Samba rpm from rhn.redhat.com
rpm -ivh samba*.rpm
Configuring Samba:
cd /etc/samba
mv smb.conf smb.conf.backup
vi smb.conf
[global]
workgroup = wrkgrp
netbios name = smbserver
security = SHARE
load printers = No
default service = global
path = /home
available = No
encrypt passwords = yes
[share]
writeable = yes
admin users = smbuser
path = /home/share
force user = root
valid users = smbuser
public = yes
available = yes
save and exit
adduser smbuser #add unix account
passwd smbuser #set unix account password
smbpasswd -a smbuser #lets create same user account on samba
/etc/init.d/smb restart
Now let us setup drive letter on our Windows so we can easily access these files.
Start -> run -> cmd
At the prompt type: net use z: \\ip_of_your_samba_server\share /user: smbuser password_you_assigned
That is it! At this point you have successfully set up Samba under Linux and are now successfully connected to your share from your Windows machine
Exploiting the Linux Kernel
Introduction
In Linux you run processes in two different modes of execution. There is userspace (aka user mode) which you run your everyday applications, like Firefox, Pidgin, irssi. From the kernel’s point of view, this is unprivileged mode, meaning user space applications don’t have access to hardware, or bits of the system critical to its function. The next mode is kernelspace, in this mode a process runs in privileged mode, giving it access to hardware and low-level system processes. On x86 there are four modes of execution called ‘rings’, Linux uses only two of these rings: ring 0 for kernelspace, and ring 3 for userspace.
Note: Do not confuse ‘privileges’ with those of the Linux system (e.g., root), they are two completely separate concepts.
To write a program that runs in kernel space you need to write what is known as a kernel module. A kernel module is a piece of code that is loaded and unloaded into the kernel on demand. They add functionality to the kernel without needing to restart the whole system. Without them a kernel would be known as ‘monolithic’ meaning all functionality must be directly written in the kernel image.
You can see what modules are already loaded into your kernel by running lsmod. You can load modules using the programs provided by the mod-utils package. insmod is used for loading *.ko files which will be the product of our examples.
Note: It is recommended that you do not use X to load these modules. Kernel modules cannot print to stdout using printf etc. When loading the modules do so from a console to see printk output
Hello, World!
#include /* Needed by all modules */
#include /* Needed for KERN_ALERT */
#include /* Needed for the macros */
char *hello = "Hello, World!";
char *goodbye = "Goodbye!";
static int kmodule_init(void)
{
printk(KERN_ALERT "%s\n",hello);
return 0;
}
static void kmodule_exit(void)
{
printk(KERN_ALERT "%s\n",goodbye);
}
/* run functions */
module_init(kmodule_init);
module_exit(kmodule_exit);
Kernel modules must have at least two functions, the init function and the exit function. In this example they are called kmodule_init and kmodule_exit respectively, these can be called whatever you like. They are called by the functions module_init and module_exit, these names must remain the same.
WTF is printk?
Unlike printf(), printk() is not designed to communicate with the user, instead it is a logging system for the kernel. You can see this if you type dmesg | tail -n 2 into a console. What should appear is:
Hello, World!
Goodbye!
The bit of printk() in our example that says KERN_ALERT defines its priority. The kernel comes with 8 priorities which can be seen in linux/kernel.h. Our use of KERN_ALERT makes sure the messages are printed to the console, not just appended to /var/log/messages
Compiling kernel modules
The easiest way to properly compile kernel modules is to use make. So, in the same directory as your code is in create a Makefile with this as its only line:
obj-m += hello_world.o
Change the name of the file to match what you called your code file, make sure you use the .o file extension and not .c.
Now you can run the command make -C /usr/src/linux-`uname -r` SUBDIRS=$PWD modules to compile the necessary files. Next run (as root, on a console) insmod hello_world.ko.
If all goes well, you will see “Hello, World!” printed on your screen. Congratulations, you just successfully created your first kernel module! Now unload your module with rmmod hello_world.
Now the good stuff…
When programming in userspace and you need to make a request to the kernel you use a syscall. These are like basic userspace functions like read(), write(), and mkdir(). In kernel space there are the corresponding functions sys_read(), sys_write(), and sys_mkdir(). These are known as syscalls. These are implemented into the kernel. When mkdir() is used it “calls” the appropriate function, in this case sys_mkdir()
The mkdir function in a shell uses the mkdir() function. To do this the command must know where sys_mkdir() is located in the memory. It does this using the sys_call_table[] which holds pointers to all the system calls:
user types: mkdir code -> function: mkdir() -> sys_call_table[] -> system call: sys_mkdir() -> file system
Absolutely all kernel based rootkits use the same technique. It changes the pointer value in sys_call_table[] to point to its own function. This was quite simple in kernels previous to 2.6, you could just put this line into your code: extern void *sys_call_table[]; and include sys/syscall.h. But since the advent of 2.6 sys_call_table[] is no longer exported, so some Mad Hacking ™ is required. To find it we need to look for int 0x80 because when a syscall is issued the application is interrupted and control is given to the kernel. This is done with the instruction int 0x80. When the instruction int 0×80 is reached the kernel executes a special function: _system_call().
To find int 0x80 we need to look in the interrupt descriptor table. We do that with this bit of code:
struct {
unsigned short limit;
unsigned int base;
} __attribute__ ((packed)) idtr;
/* ask the processor for the idt address and store it in idtr */
asm ("sidt %0" : "=m" (idtr));
The address of the idt is held in a special register (idtr) there is an assembly function that can be used to obtain that address. Next we need to find int 0x80 and hence find the system_call function.
struct {
unsigned short off1;
unsigned short sel;
unsigned char none,flags;
unsigned short off2;
} __attribute__ ((packed)) idt;
unsigned sys_call_off;
/* read in IDT for int 0x80 (syscall) */
memcpy(&idt, idtr.base+8*0x80,sizeof(idt));
sys_call_off = (idt.off2 << 16) | idt.off1;
Each entry in the idt is 8 bytes (hence the base+8*0x80) and the magic shifting of bits to reconstruct the location of system_call() used below.
Now the most complex bit, we need to look at the system_call function in detail.
% gdb -q /usr/src/linux/vmlinux
(no debugging symbols found)…(gdb) disass system_call
Dump of assembler code for function system_call:
0xc0106bc8 : push %eax
0xc0106bc9 : cld
0xc0106bca : push %es
0xc0106bcb : push %ds
0xc0106bcc : push %eax
0xc0106bcd : push %ebp
0xc0106bce : push %edi
0xc0106bcf : push %esi
0xc0106bd0 : push %edx
0xc0106bd1 : push %ecx
0xc0106bd2 : push %ebx
0xc0106bd3 : mov $0×18,%edx
0xc0106bd8 : mov %edx,%ds
0xc0106bda : mov %edx,%es
0xc0106bdc : mov $0xffffe000,%ebx
0xc0106be1 : and %esp,%ebx
0xc0106be3 : cmp $0×100,%eax
0xc0106be8 : jae 0xc0106c75
0xc0106bee : testb $0×2,0×18(%ebx)
0xc0106bf2 : jne 0xc0106c48
0xc0106bf4 : call *0xc01e0f18(,%eax,4) <-- that's it
0xc0106bfb : mov %eax,0×18(%esp,1)
0xc0106bff : nop
End of assembler dump.
(gdb) print &sys_call_table
$1 = ( *) 0xc01e0f18 <-- see ? it's same
(gdb) x/xw (system_call+44)
0xc0106bf4 : 0×188514ff <-- opcode (little endian)
(gdb)
What we see from the above output is the actual call to a specific syscall. The line marked <-- that's it shows that call (,%eax,4) where matches that of the sys_call_table (seen a few lines later). To find the address of the sys_call_table we’ll start at the system_call() function and inspect memory until we find something that looks like call (,%eax,4) and assume that is the sys_call_table. The opcode for call is shown at the end of the above output. Thus, our code becomes:
char *p;
unsigned sys_call_table;
p = (char*)memmem (sc_asm,CALLOFF,"\xff\x14\x85",3);
sys_call_table = *(unsigned*)(p+3);
And there you have it, a successful exportation of the sys_call_table. A clearer example is in the attached tarball. What do we do with it, you might ask. To hijack some syscalls we need the appropriate header files, so we know the correct syscall numbers. That file is linux/unistd.h, it includes the definitions for syscall numbers with the naming convention: __NR_name. In this example we are changing the sys_open call. To do that we use this snippet:
asmlinkage int (*original_sys_open)(const char *, int, int);
asmlinkage int new_sys_open(const char *filename, int flags, int mode)
{
int ret;
printk(KERN_ALERT "CALLING SYSOPEN!\n");
/* let the real sys_open do the rest */
ret = (*original_sys_open)(filename, flags, mode);
return ret;
}
[...]
original_sys_open = sct[__NR_open];
printk(KERN_ALERT "Syscall table @ 0x%x\n", sct);
printk(KERN_ALERT "ORIG: 0x%x \n", original_sys_open);
printk(KERN_ALERT "addr of __NR_open: 0x%x \n", &sct[__NR_open]);
sct[__NR_open] = new_sys_open; /* redefine with our own call */
printk(KERN_ALERT "DONE\n");
In the above example we simply patch the sys_open call to print a small message whenever it’s used and then passes execution on to the original sys_open. There are also some more creative uses for this, like logging a user’s actions by hijacking sys_read, or hiding files from being shown when a user runs ls
That just about wraps it up, attached below are the full examples with working Makefiles etc.
In Linux you run processes in two different modes of execution. There is userspace (aka user mode) which you run your everyday applications, like Firefox, Pidgin, irssi. From the kernel’s point of view, this is unprivileged mode, meaning user space applications don’t have access to hardware, or bits of the system critical to its function. The next mode is kernelspace, in this mode a process runs in privileged mode, giving it access to hardware and low-level system processes. On x86 there are four modes of execution called ‘rings’, Linux uses only two of these rings: ring 0 for kernelspace, and ring 3 for userspace.
Note: Do not confuse ‘privileges’ with those of the Linux system (e.g., root), they are two completely separate concepts.
To write a program that runs in kernel space you need to write what is known as a kernel module. A kernel module is a piece of code that is loaded and unloaded into the kernel on demand. They add functionality to the kernel without needing to restart the whole system. Without them a kernel would be known as ‘monolithic’ meaning all functionality must be directly written in the kernel image.
You can see what modules are already loaded into your kernel by running lsmod. You can load modules using the programs provided by the mod-utils package. insmod is used for loading *.ko files which will be the product of our examples.
Note: It is recommended that you do not use X to load these modules. Kernel modules cannot print to stdout using printf etc. When loading the modules do so from a console to see printk output
Hello, World!
#include
#include
#include
char *hello = "Hello, World!";
char *goodbye = "Goodbye!";
static int kmodule_init(void)
{
printk(KERN_ALERT "%s\n",hello);
return 0;
}
static void kmodule_exit(void)
{
printk(KERN_ALERT "%s\n",goodbye);
}
/* run functions */
module_init(kmodule_init);
module_exit(kmodule_exit);
Kernel modules must have at least two functions, the init function and the exit function. In this example they are called kmodule_init and kmodule_exit respectively, these can be called whatever you like. They are called by the functions module_init and module_exit, these names must remain the same.
WTF is printk?
Unlike printf(), printk() is not designed to communicate with the user, instead it is a logging system for the kernel. You can see this if you type dmesg | tail -n 2 into a console. What should appear is:
Hello, World!
Goodbye!
The bit of printk() in our example that says KERN_ALERT defines its priority. The kernel comes with 8 priorities which can be seen in linux/kernel.h. Our use of KERN_ALERT makes sure the messages are printed to the console, not just appended to /var/log/messages
Compiling kernel modules
The easiest way to properly compile kernel modules is to use make. So, in the same directory as your code is in create a Makefile with this as its only line:
obj-m += hello_world.o
Change the name of the file to match what you called your code file, make sure you use the .o file extension and not .c.
Now you can run the command make -C /usr/src/linux-`uname -r` SUBDIRS=$PWD modules to compile the necessary files. Next run (as root, on a console) insmod hello_world.ko.
If all goes well, you will see “Hello, World!” printed on your screen. Congratulations, you just successfully created your first kernel module! Now unload your module with rmmod hello_world.
Now the good stuff…
When programming in userspace and you need to make a request to the kernel you use a syscall. These are like basic userspace functions like read(), write(), and mkdir(). In kernel space there are the corresponding functions sys_read(), sys_write(), and sys_mkdir(). These are known as syscalls. These are implemented into the kernel. When mkdir() is used it “calls” the appropriate function, in this case sys_mkdir()
The mkdir function in a shell uses the mkdir() function. To do this the command must know where sys_mkdir() is located in the memory. It does this using the sys_call_table[] which holds pointers to all the system calls:
user types: mkdir code -> function: mkdir() -> sys_call_table[] -> system call: sys_mkdir() -> file system
Absolutely all kernel based rootkits use the same technique. It changes the pointer value in sys_call_table[] to point to its own function. This was quite simple in kernels previous to 2.6, you could just put this line into your code: extern void *sys_call_table[]; and include sys/syscall.h. But since the advent of 2.6 sys_call_table[] is no longer exported, so some Mad Hacking ™ is required. To find it we need to look for int 0x80 because when a syscall is issued the application is interrupted and control is given to the kernel. This is done with the instruction int 0x80. When the instruction int 0×80 is reached the kernel executes a special function: _system_call().
To find int 0x80 we need to look in the interrupt descriptor table. We do that with this bit of code:
struct {
unsigned short limit;
unsigned int base;
} __attribute__ ((packed)) idtr;
/* ask the processor for the idt address and store it in idtr */
asm ("sidt %0" : "=m" (idtr));
The address of the idt is held in a special register (idtr) there is an assembly function that can be used to obtain that address. Next we need to find int 0x80 and hence find the system_call function.
struct {
unsigned short off1;
unsigned short sel;
unsigned char none,flags;
unsigned short off2;
} __attribute__ ((packed)) idt;
unsigned sys_call_off;
/* read in IDT for int 0x80 (syscall) */
memcpy(&idt, idtr.base+8*0x80,sizeof(idt));
sys_call_off = (idt.off2 << 16) | idt.off1;
Each entry in the idt is 8 bytes (hence the base+8*0x80) and the magic shifting of bits to reconstruct the location of system_call() used below.
Now the most complex bit, we need to look at the system_call function in detail.
% gdb -q /usr/src/linux/vmlinux
(no debugging symbols found)…(gdb) disass system_call
Dump of assembler code for function system_call:
0xc0106bc8 : push %eax
0xc0106bc9 : cld
0xc0106bca : push %es
0xc0106bcb : push %ds
0xc0106bcc : push %eax
0xc0106bcd : push %ebp
0xc0106bce : push %edi
0xc0106bcf : push %esi
0xc0106bd0 : push %edx
0xc0106bd1 : push %ecx
0xc0106bd2 : push %ebx
0xc0106bd3 : mov $0×18,%edx
0xc0106bd8 : mov %edx,%ds
0xc0106bda : mov %edx,%es
0xc0106bdc : mov $0xffffe000,%ebx
0xc0106be1 : and %esp,%ebx
0xc0106be3 : cmp $0×100,%eax
0xc0106be8 : jae 0xc0106c75
0xc0106bee : testb $0×2,0×18(%ebx)
0xc0106bf2 : jne 0xc0106c48
0xc0106bf4 : call *0xc01e0f18(,%eax,4) <-- that's it
0xc0106bfb : mov %eax,0×18(%esp,1)
0xc0106bff : nop
End of assembler dump.
(gdb) print &sys_call_table
$1 = ( *) 0xc01e0f18 <-- see ? it's same
(gdb) x/xw (system_call+44)
0xc0106bf4 : 0×188514ff <-- opcode (little endian)
(gdb)
What we see from the above output is the actual call to a specific syscall. The line marked <-- that's it shows that call (,%eax,4) where matches that of the sys_call_table (seen a few lines later). To find the address of the sys_call_table we’ll start at the system_call() function and inspect memory until we find something that looks like call (,%eax,4) and assume that is the sys_call_table. The opcode for call is shown at the end of the above output. Thus, our code becomes:
char *p;
unsigned sys_call_table;
p = (char*)memmem (sc_asm,CALLOFF,"\xff\x14\x85",3);
sys_call_table = *(unsigned*)(p+3);
And there you have it, a successful exportation of the sys_call_table. A clearer example is in the attached tarball. What do we do with it, you might ask. To hijack some syscalls we need the appropriate header files, so we know the correct syscall numbers. That file is linux/unistd.h, it includes the definitions for syscall numbers with the naming convention: __NR_name. In this example we are changing the sys_open call. To do that we use this snippet:
asmlinkage int (*original_sys_open)(const char *, int, int);
asmlinkage int new_sys_open(const char *filename, int flags, int mode)
{
int ret;
printk(KERN_ALERT "CALLING SYSOPEN!\n");
/* let the real sys_open do the rest */
ret = (*original_sys_open)(filename, flags, mode);
return ret;
}
[...]
original_sys_open = sct[__NR_open];
printk(KERN_ALERT "Syscall table @ 0x%x\n", sct);
printk(KERN_ALERT "ORIG: 0x%x \n", original_sys_open);
printk(KERN_ALERT "addr of __NR_open: 0x%x \n", &sct[__NR_open]);
sct[__NR_open] = new_sys_open; /* redefine with our own call */
printk(KERN_ALERT "DONE\n");
In the above example we simply patch the sys_open call to print a small message whenever it’s used and then passes execution on to the original sys_open. There are also some more creative uses for this, like logging a user’s actions by hijacking sys_read, or hiding files from being shown when a user runs ls
That just about wraps it up, attached below are the full examples with working Makefiles etc.
Wednesday, August 15, 2007
install DHCP on Linux
First off, a definition. DHCP is the Dynamic Host Configuration Protocol and rather than use the default DHCP server included with your Red Hat / Fedora Linux system, I recommend that you pop over to the Internet Software Consortium and Download the latest version of the DHCP Server. As I write this, it's at version 3.0.3.
Why is it important to have the latest version? In a word, security. As with any software that you're going to run on your server, it's critically important that you have the very latest version of this 'daemon' (as we Linux geeks call programs that run on the server without intervention) on your system. It's also very important to shut off any services you aren't using (a process I detail in my popular book Teach Yourself Unix System Administration in 24 Hours, by the way).
So once you've downloaded the latest version of the software, you'll want to unpack it with:
$ tar xzf ./dhcp-303-tar.gz
Now, move to the new directory that contains all the source and type in the following commands (the tons of output these commands have has been deleted to save our sanity here):
$ ./configure
$ make
Assuming all has gone well, switch to root by using the sudo command and install the new server:
$ sudo make install
you'll be prompted for the root password, then, if you typed it in correctly, the new DHCP server will be installed onto your system.
Good. Now you have the latest DHCP server it's time to configure it properly for your environment. This is best done by copying the file server/dhcp.conf from the installation directory into your /etc directory, like this:
$ sudo cp server/dhcp.conf /etc
This time, since you just did a sudo command a few seconds ago, you won't be prompted for your password (an exceedingly slick feature of sudo, actually!)
Now, again using sudo, it's time to edit the configuration file to match your system configuration. Here's what the dhcp.conf file looks like:
ddns-update-style interim # Redhat Version 8.0+
subnet 192.168.1.0 netmask 255.255.255.0 {
# The range of IP addresses the server will issue to
# DHCP enabled PC clients booting up on the network
range 192.168.1.201 192.168.1.220;
# Set the amount of time in seconds that
# a client may keep the IP address
default-lease-time 86400;
max-lease-time 86400;
# Set the default gateway to be used by
# the PC clients
option routers 192.168.1.1;
# Don't forward DHCP requests from this NIC interface
# to any other NIC interfaces
option ip-forwarding off;
# Set the broadcast address and subnet mask
# to be used by the DHCP clients
option broadcast-address 192.168.1.255;
option subnet-mask 255.255.255.0;
# Set the DNS server to be used by the
# DHCP clients
option domain-name-servers 192.168.1.100;
# Set the NTP server to be used by the
# DHCP clients
option nntp-server 192.168.1.100;
# If you specify a WINS server for your Windows clients,
# you need to include the following option in the dhcpd.conf file:
option netbios-name-servers 192.168.1.100;
}
# List an unused interface here
#
subnet 192.168.2.0 netmask 255.255.255.0 {
}
# You can also assign specific IP addresses based on the clients'
# ethernet MAC address as follows (Host's name is "smallfry":
host smallfry {
hardware ethernet 08:00:2b:4c:59:23;
fixed-address 192.168.1.222;
}
As with many Linux configuration files, this is actually fairly well documented, especially since you should be able to type man dhcp-options to get an exhaustive explanation of each and every configuration option.
In particular, make sure you set the domain name properly, identify your set of DNS servers by name, and define the subnet range for which you want to provide services via DHCP.
Once that's all configured to your liking, a little bit more tweaking is required to get everything checked and started properly:
$ sudo touch /var/lib/dhcp/dhcp.leases
Will make sure that you have a 'leases' file, a critical part of how the DHCP server tracks what systems it's seen and serviced.
$ sudo chkconfig --level 35 dhcpd on
This will check your configuration and make sure it will be added to the list of daemons to start up at boot time from now on. Very useful if you don't want to remember to restart it each time!
$ /etc/init.d/dhcp restart
Now you should be running a DHCP server on your system. Check to make sure it's running by using ps aux | grep dhcp but you should be
HAYA TUTONANA MUNGU AKIPENDA...
Why is it important to have the latest version? In a word, security. As with any software that you're going to run on your server, it's critically important that you have the very latest version of this 'daemon' (as we Linux geeks call programs that run on the server without intervention) on your system. It's also very important to shut off any services you aren't using (a process I detail in my popular book Teach Yourself Unix System Administration in 24 Hours, by the way).
So once you've downloaded the latest version of the software, you'll want to unpack it with:
$ tar xzf ./dhcp-303-tar.gz
Now, move to the new directory that contains all the source and type in the following commands (the tons of output these commands have has been deleted to save our sanity here):
$ ./configure
$ make
Assuming all has gone well, switch to root by using the sudo command and install the new server:
$ sudo make install
you'll be prompted for the root password, then, if you typed it in correctly, the new DHCP server will be installed onto your system.
Good. Now you have the latest DHCP server it's time to configure it properly for your environment. This is best done by copying the file server/dhcp.conf from the installation directory into your /etc directory, like this:
$ sudo cp server/dhcp.conf /etc
This time, since you just did a sudo command a few seconds ago, you won't be prompted for your password (an exceedingly slick feature of sudo, actually!)
Now, again using sudo, it's time to edit the configuration file to match your system configuration. Here's what the dhcp.conf file looks like:
ddns-update-style interim # Redhat Version 8.0+
subnet 192.168.1.0 netmask 255.255.255.0 {
# The range of IP addresses the server will issue to
# DHCP enabled PC clients booting up on the network
range 192.168.1.201 192.168.1.220;
# Set the amount of time in seconds that
# a client may keep the IP address
default-lease-time 86400;
max-lease-time 86400;
# Set the default gateway to be used by
# the PC clients
option routers 192.168.1.1;
# Don't forward DHCP requests from this NIC interface
# to any other NIC interfaces
option ip-forwarding off;
# Set the broadcast address and subnet mask
# to be used by the DHCP clients
option broadcast-address 192.168.1.255;
option subnet-mask 255.255.255.0;
# Set the DNS server to be used by the
# DHCP clients
option domain-name-servers 192.168.1.100;
# Set the NTP server to be used by the
# DHCP clients
option nntp-server 192.168.1.100;
# If you specify a WINS server for your Windows clients,
# you need to include the following option in the dhcpd.conf file:
option netbios-name-servers 192.168.1.100;
}
# List an unused interface here
#
subnet 192.168.2.0 netmask 255.255.255.0 {
}
# You can also assign specific IP addresses based on the clients'
# ethernet MAC address as follows (Host's name is "smallfry":
host smallfry {
hardware ethernet 08:00:2b:4c:59:23;
fixed-address 192.168.1.222;
}
As with many Linux configuration files, this is actually fairly well documented, especially since you should be able to type man dhcp-options to get an exhaustive explanation of each and every configuration option.
In particular, make sure you set the domain name properly, identify your set of DNS servers by name, and define the subnet range for which you want to provide services via DHCP.
Once that's all configured to your liking, a little bit more tweaking is required to get everything checked and started properly:
$ sudo touch /var/lib/dhcp/dhcp.leases
Will make sure that you have a 'leases' file, a critical part of how the DHCP server tracks what systems it's seen and serviced.
$ sudo chkconfig --level 35 dhcpd on
This will check your configuration and make sure it will be added to the list of daemons to start up at boot time from now on. Very useful if you don't want to remember to restart it each time!
$ /etc/init.d/dhcp restart
Now you should be running a DHCP server on your system. Check to make sure it's running by using ps aux | grep dhcp but you should be
HAYA TUTONANA MUNGU AKIPENDA...
Host Performance Monitoring Using 10G Enterprise Manager Grid Control
Host Performance Monitoring Using 10G Enterprise Manager Grid Control No matter how expertly you tune an Oracle database, if the server it runs on is overloaded, underpowered or configured incorrectly, you will never obtain "maximus database performis".
So, let's start our discussion on 10G Enterprise Manager's performance monitoring features by investigating the tool's new host monitoring capabilities. This first blog will provide you with a quick tour of EM's hot new host monitoring component. In my next blog, I'll show you how you use the information it generates to monitor operating system and hardware performance.10G Enterprise Manager calls objects that it monitors and administers "targets." Target types include but are not limited to hosts, databases, application servers, listeners, and third-party applications. 10G EM's home page allows administrators to easily navigate to the different target types. This blog will focus on the host target type.
Host Home Page
If you take a look at the host's home page you'll notice a configuration section on the right hand side of the page. This section contains navigation links that allow administrators to view hardware and operating system configuration parameters. This system-related information, often only available to O/S administrators, allows DBAs to quickly view parameters that affect server and database performance. The drill-down panels also include information on O/S packages, and patches, disk configuration and file system freespace.
At the bottom of the host's home page is a section that display's information on alerts that are currently active. Alerts are generated when a metric's threshold is exceeded. Metrics are units of measurement that are set at predefined values to assess the health of the target being monitored. These predefined values are called metric thresholds. Thresholds are one or more values against which current statistical values are compared.
10G EM divides the threshold settings into warning and critical categories and allows administrators to configure a specific set of actions that will occur when that threshold category is reached. For example, the DBA may want to be e-mailed if a particular host's CPU utilization exceeds the warning threshold of 70% and paged if the host's CPU utilization exceeds the critical threshold of 90%.
Each target (host, database, listener, application server) has its own set of predefined metrics that can be customized to address each application's unique performance monitoring requirements. In addition, administrators are able to create their own set of user-defined metrics to provide a truly customized monitoring strategy for each monitored target. We'll discuss how administrators define and administer metrics and thresholds in another series of blogs.
Each host's home page contains tabs that allow users to navigate to Performance, Targets and Configuration panels. Let's take a look at each one.
Performance Tab
The host's performance panel provides information on current CPU, Memory and Disk resource utilization. Each resource's graphical display contains links that allow users to view more detailed statistical information. Each of the drill down panels for CPU, Memoryand Disk contains a drop down menu list that allows users to view current as well as historical performance statistics for the last 24 hours, 7 days and 31 days.
The main performance panel also displays information on the top resource consuming processes currently running on the host. Once again, a drop down menu allows administrators to switch between top CPU and top memory consumption reports. We'll learn how to use this report to speed the performance problem resolution process in the next blog.
Target Tab
If you click on the Targets tab on the host's main panel, a listing of the dependent or sub-targets (listeners, application servers, databases, etc) running on that host is provided. The links allow administrators to easily navigate to the desired target.
Configuration Tab
Clicking on the configuration tab on the host's main panel sends users to the main configuration display screen. The configuration panel displays information about the hosts' hardware and software configuration.
The information includes:
Hardware Configuration - memory, CPU, I/O and network interface configuration information
Operating System Configuration - release levels, kernel parameters and patches
Third-party package software installations - (i.e. Veritas data storage management system packages, EMC drivers, etc.)
Oracle software installations - products, components and patch sets
The host's configuration panel has an option to capture key configuration values to a flat file which can be viewed as a report or used in host configuration comparisons. Although I haven't needed it yet, it would seem that comparing host configurations would be a pretty useful function. Enterprise manager supports the following host configuration comparisons:
Two host configurations
Two host configuration files
Host configuration and configuration file
One host configuration compared to multiple hosts
All aforementioned comparisons above can be done interactively except comaparisons between one host to multiple hosts which must be performed using 10G EM's batch job system. The summary results of the comparison are presented in a tabular format. The report contains links which drill down into more detailed comparison information.
Next Up
Now that we have a better understanding of some of the host information that 10G EM provides , I'll provide you with a few helpful hints on using this information to determine if the hardware platform and operating combination are helping (or hurting) database performance. Thanks for reading..
So, let's start our discussion on 10G Enterprise Manager's performance monitoring features by investigating the tool's new host monitoring capabilities. This first blog will provide you with a quick tour of EM's hot new host monitoring component. In my next blog, I'll show you how you use the information it generates to monitor operating system and hardware performance.10G Enterprise Manager calls objects that it monitors and administers "targets." Target types include but are not limited to hosts, databases, application servers, listeners, and third-party applications. 10G EM's home page allows administrators to easily navigate to the different target types. This blog will focus on the host target type.
Host Home Page
If you take a look at the host's home page you'll notice a configuration section on the right hand side of the page. This section contains navigation links that allow administrators to view hardware and operating system configuration parameters. This system-related information, often only available to O/S administrators, allows DBAs to quickly view parameters that affect server and database performance. The drill-down panels also include information on O/S packages, and patches, disk configuration and file system freespace.
At the bottom of the host's home page is a section that display's information on alerts that are currently active. Alerts are generated when a metric's threshold is exceeded. Metrics are units of measurement that are set at predefined values to assess the health of the target being monitored. These predefined values are called metric thresholds. Thresholds are one or more values against which current statistical values are compared.
10G EM divides the threshold settings into warning and critical categories and allows administrators to configure a specific set of actions that will occur when that threshold category is reached. For example, the DBA may want to be e-mailed if a particular host's CPU utilization exceeds the warning threshold of 70% and paged if the host's CPU utilization exceeds the critical threshold of 90%.
Each target (host, database, listener, application server) has its own set of predefined metrics that can be customized to address each application's unique performance monitoring requirements. In addition, administrators are able to create their own set of user-defined metrics to provide a truly customized monitoring strategy for each monitored target. We'll discuss how administrators define and administer metrics and thresholds in another series of blogs.
Each host's home page contains tabs that allow users to navigate to Performance, Targets and Configuration panels. Let's take a look at each one.
Performance Tab
The host's performance panel provides information on current CPU, Memory and Disk resource utilization. Each resource's graphical display contains links that allow users to view more detailed statistical information. Each of the drill down panels for CPU, Memoryand Disk contains a drop down menu list that allows users to view current as well as historical performance statistics for the last 24 hours, 7 days and 31 days.
The main performance panel also displays information on the top resource consuming processes currently running on the host. Once again, a drop down menu allows administrators to switch between top CPU and top memory consumption reports. We'll learn how to use this report to speed the performance problem resolution process in the next blog.
Target Tab
If you click on the Targets tab on the host's main panel, a listing of the dependent or sub-targets (listeners, application servers, databases, etc) running on that host is provided. The links allow administrators to easily navigate to the desired target.
Configuration Tab
Clicking on the configuration tab on the host's main panel sends users to the main configuration display screen. The configuration panel displays information about the hosts' hardware and software configuration.
The information includes:
Hardware Configuration - memory, CPU, I/O and network interface configuration information
Operating System Configuration - release levels, kernel parameters and patches
Third-party package software installations - (i.e. Veritas data storage management system packages, EMC drivers, etc.)
Oracle software installations - products, components and patch sets
The host's configuration panel has an option to capture key configuration values to a flat file which can be viewed as a report or used in host configuration comparisons. Although I haven't needed it yet, it would seem that comparing host configurations would be a pretty useful function. Enterprise manager supports the following host configuration comparisons:
Two host configurations
Two host configuration files
Host configuration and configuration file
One host configuration compared to multiple hosts
All aforementioned comparisons above can be done interactively except comaparisons between one host to multiple hosts which must be performed using 10G EM's batch job system. The summary results of the comparison are presented in a tabular format. The report contains links which drill down into more detailed comparison information.
Next Up
Now that we have a better understanding of some of the host information that 10G EM provides , I'll provide you with a few helpful hints on using this information to determine if the hardware platform and operating combination are helping (or hurting) database performance. Thanks for reading..
Tuesday, August 14, 2007
installing forms6i on XP gives error
PURPOSE
-------
The purpose of this document is to aid customers install Forms & Reports 6i
over MS Windows XP Operating System.
Please notice that this installation is not certified or supported at the moment.
The earliest certified release could be Developer Forms & Reports 6i patchset 13,
however this is not guaranteed at the time of writing.
Note that it is certified for a Forms6i Client (web deployed) to run on XP.
At the date of this article the following client certifications apply for XP:
Oracle JInitiator (Win32(1), based on JDK 1.1.8.x) - Patch set 10
Oracle JInitiator (Win32(1), based on JDK 1.3.x) - Patch set 10
Microsoft Internet Explorer 6.0 (Microsoft VM 5.0) - Patch set 13
JavaSoft 1.4.1x Java Plug-in - Patch set 12
SCOPE & APPLICATION
-------------------
This is strictly an installation guide to overcome issues with the
Oracle Installer (ORCA Version 3.3.1.2.4 shipped with Oracle Developer
Forms & Reports 6i CD) The Oracle Installer has built in checks for what is
assumes to be a Windows NT 4.0 environment.
The procedure described in this document utilizes the Windows XP compatibility
Mode to trick Oracle Installer so that it will think
that it is running under MS Windows NT 4.0 with Service Pack 5 instead of the
actual host platform MS Windows XP.
HOW TO INSTALL FORMS & REPORTS 6i OVER MS WINDOWS XP OPERATION SYSTEM
---------------------------------------------------------------------
1. Enter the Forms & Reports 6i Release 2 CDROM in the drive.
2. Run the Program Compatibility Wizard
From Start Menu->Programs->Accessories->Program Compatibility Wizard
3. Click the following:
Next->I want to use the program in the CDROM drive->
Next-> Microsoft Windows NT Service Pack 5->
Next->Next->
Next-> Yes, set this program to always use this compatibility settings->
Next->No->Next->Finish
4. During the steps taken during the Program Compatibility Wizard the test
screen actually runs up Oracle Installer and it is possible to perform the
installation at this stage)
If the decision was taken not to do this at this stage, perform the
installation now by double-clicking on setup.exe.
5. Note having run through the above steps installing any product via the CD
ROM Windows XP will remember to run in Windows NT 4.0 compatibility mode.
6. It is recommended to apply the latest patchset after the installation of
Oracle Developer Forms & Reports 6i on Windows XP.
In order to be able to also apply the patchset succesfully, the Program
Compatibiltity Wizard again needs to be invoked.
a. Specify 'I want to locate the program manually',
b. Browse for orainst.exe within the\INSTALL
c. Set compatibility to be 'MS Windows NT 4.0 with Service Pack 5'.
d. Again either perform the installation during the compatibility test step
or run orainst.exe from\INSTALL directory
after compatibility has been set.
-------
The purpose of this document is to aid customers install Forms & Reports 6i
over MS Windows XP Operating System.
Please notice that this installation is not certified or supported at the moment.
The earliest certified release could be Developer Forms & Reports 6i patchset 13,
however this is not guaranteed at the time of writing.
Note that it is certified for a Forms6i Client (web deployed) to run on XP.
At the date of this article the following client certifications apply for XP:
Oracle JInitiator (Win32(1), based on JDK 1.1.8.x) - Patch set 10
Oracle JInitiator (Win32(1), based on JDK 1.3.x) - Patch set 10
Microsoft Internet Explorer 6.0 (Microsoft VM 5.0) - Patch set 13
JavaSoft 1.4.1x Java Plug-in - Patch set 12
SCOPE & APPLICATION
-------------------
This is strictly an installation guide to overcome issues with the
Oracle Installer (ORCA Version 3.3.1.2.4 shipped with Oracle Developer
Forms & Reports 6i CD) The Oracle Installer has built in checks for what is
assumes to be a Windows NT 4.0 environment.
The procedure described in this document utilizes the Windows XP compatibility
Mode to trick Oracle Installer so that it will think
that it is running under MS Windows NT 4.0 with Service Pack 5 instead of the
actual host platform MS Windows XP.
HOW TO INSTALL FORMS & REPORTS 6i OVER MS WINDOWS XP OPERATION SYSTEM
---------------------------------------------------------------------
1. Enter the Forms & Reports 6i Release 2 CDROM in the drive.
2. Run the Program Compatibility Wizard
From Start Menu->Programs->Accessories->Program Compatibility Wizard
3. Click the following:
Next->I want to use the program in the CDROM drive->
Next-> Microsoft Windows NT Service Pack 5->
Next->Next->
Next-> Yes, set this program to always use this compatibility settings->
Next->No->Next->Finish
4. During the steps taken during the Program Compatibility Wizard the test
screen actually runs up Oracle Installer and it is possible to perform the
installation at this stage)
If the decision was taken not to do this at this stage, perform the
installation now by double-clicking on setup.exe.
5. Note having run through the above steps installing any product via the CD
ROM Windows XP will remember to run in Windows NT 4.0 compatibility mode.
6. It is recommended to apply the latest patchset after the installation of
Oracle Developer Forms & Reports 6i on Windows XP.
In order to be able to also apply the patchset succesfully, the Program
Compatibiltity Wizard again needs to be invoked.
a. Specify 'I want to locate the program manually',
b. Browse for orainst.exe within the
c. Set compatibility to be 'MS Windows NT 4.0 with Service Pack 5'.
d. Again either perform the installation during the compatibility test step
or run orainst.exe from
after compatibility has been set.
Oracle on LInux (Backup Strategy)
Backup Strategy
The purpose of backup is to enable recovery. Backup and recovery are always risky; here are some steps that minimize the chance recovery is necessary:
Store everything on a fault-tolerant disk array (RAID 1 or 5 or better).
Use battery backup.
Use more reliable hardware, such as SCSI instead of IDE.
These steps improve the chances of successful recovery:
Store backups on a third disk on another controller
Store backups on a different computer on a different network in a different physical location. (Compared to off-line backup such as tapes and CDRs, on-line backup is faster and more likely to succeed, but requires maintenance of another machine.)
Plan and configure for recovery from the beginning.
Test your recovery strategy from time to time.
Make it easy to maintain and test your recovery strategy, so that you are more likely to do it.
OpenACS installations comprise files and database contents. If you follow the reference install and put all files, including configuration files, in /web/servicename/, and back up the database nightly to a file in /web/servicename/database-backup, then you can apply standard file-based backup strategies to the root directory.
Set up Nightly Postgres Exports
A quick way to automate database backup is a cron job. (This should moved into OpenACS's scheduled task project so that it's integrated with OpenACS's alerts and such.)
[service0@yourserver service0]$ export EDITOR=emacs;crontab -eAdd this line to the file. The numbers and stars at the beginning are cron columns that specify when the program should be run - in this case, whenever the minute is 0 and the hour is 1, i.e., 1:00 am every day.
0 1 * * * /usr/local/pgsql/bin/pg_dump -f /web/service0/database-backup/service0_$(date +%Y-%m-%d).dmp service0Back up the file system
Here's a quick manual way to back up a reference install - it should be replaced by an automated script within OpenACS. The command excludes the auto-generated supervise directory, which is unneccesary and has complicated permissions. Make sure that you are using the cron job to back up the database to a file in /web/service0/database-backup so that the tar command will include the database.
[root@yourserver root]# su - service0
[service0@yourserver service0]$ tar -cpsj --exclude /web/service0/etc/daemontools/supervise --file /tmp/service0-backup.tar.bz2 /web/service0/
tar: Removing leading `/' from member names
[service0@yourserver service0]$Testing
On a test service, make sure that your backup-recovery process work. After backing up the database and file system, delete the service as detailed below and then recover it.
[root@yourserver root]# svc -d /service/service0
[root@yourserver root]# mv /web/service0/ /web/service0.lost
[root@yourserver root]# rm /service/service0
rm: remove symbolic link `/service/service0'? y
[root@yourserver root]# ps -auxw | grep service0
root 1496 0.0 0.0 1312 252 ? S 16:58 0:00 supervise service0
[root@yourserver root]# kill 1496
[root@yourserver root]# ps -auxw | grep service0
[root@yourserver root]# su - postgres
[postgres@yourserver pgsql]$ dropdb service0
DROP DATABASE
[postgres@yourserver pgsql]$ dropuser service0
DROP USER
[postgres@yourserver pgsql]$ exit
logout
[root@yourserver root]#Recovery
Restore the operating system and required software. You can do this with standard backup processes or by keeping copies of the install material (OS CDs, OpenACS tarball and supporting software) and repeating the install guide.
Restore the OpenACS service. Assuming the user already exists, restore the database and files from backup and restore the daemontools link. (Because of a bug in Postgres backup-recovery, not all database objects are created in the correct order. To compensate, pre-creating some objects usually work.)
[root@yourserver root]# su - postgres
[postgres@yourserver pgsql]$ createuser service0
Shall the new user be allowed to create databases? (y/n) y
Shall the new user be allowed to create more new users? (y/n) y
CREATE USER
[postgres@yourserver pgsql]$ exit
logout
[root@yourserver root]# su - service0
[service0@yourserver service0]$ cd /web
[service0@yourserver web]$ tar xjf /tmp/service0-backup.tar.bz2
[service0@yourserver web]$ chmod -R 700 service0
[service0@yourserver web]$ createdb service0
CREATE DATABASE
[service0@yourserver web]$ psql -f /web/service0/packages/acs-kernel/sql/postgresql/postgresql.sql service0
(many lines omitted)
[service0@yourserver web]$ psql service0 < /web/service0/database-backup/database-backup.dmp
(many lines omitted)
[service0@yourserver web]$ exit
[root@yourserver root]# ln -s /web/service0/etc/daemontools /service/service0
[root@yourserver root]# sleep 10
[root@yourserver root]# svgroup web /service/service0
[root@yourserver root]#Other Backup Strategies
Earlier strategies, included here because this section hasn't been fully updated yet.
Set Up Nightly Oracle Exports
(This has not yet been updated to fit with the Reference install. To do so, edit the backup script to save the backup file in /web/servicename/database-backup). While you're working with Oracle, you should configure it to do automatic exports. An export is a separate backup copy of the database. This copy includes all of the database's state at the time that the export was initiated. If your database is corrupted, you can restore from one of these backups. You should do this step as root.
Download the backup script. Save the file export-oracle.txt as /tmp/export-oracle.txt
Login as root. The following commands will install the export script:
joeuser:~$ su -
Password: ***********
root:~# cp /tmp/export-oracle.txt /usr/sbin/export-oracle
root:~# chmod 700 /usr/sbin/export-oracleSetup the export directory; this is the directory where backups will be stored. We recommend the directory /ora8/m02/oracle-exports.
root:~# mkdir /ora8/m02/oracle-exports
root:~# chown oracle.dba /ora8/m02/oracle-exports
root:~# chmod 770 /ora8/m02/oracle-exportsNow edit /usr/sbin/export-oracle and change the SERVICE_NAME and DATABASE_PASSWORD fields to their correct values. If you want to use a directory other than /ora8/m02/oracle-exports, you also need to change the exportdir setting.
Test the export procedure by running the command:
root:~# /usr/sbin/export-oracle
mv: /ora8/m02/oracle-exports/oraexport-service_name.dmp.gz: No such file or directory
Export: Release 8.1.6.1.0 - Production on Sun Jun 11 18:07:45 2000
(c) Copyright 1999 Oracle Corporation. All rights reserved.
Connected to: Oracle8i Enterprise Edition Release 8.1.6.1.0 - Production
With the Partitioning option
JServer Release 8.1.6.0.0 - Production
Export done in US7ASCII character set and US7ASCII NCHAR character set
. exporting pre-schema procedural objects and actions
. exporting foreign function library names for user SERVICE_NAME
. exporting object type definitions for user SERVICE_NAME
About to export SERVICE_NAME's objects ...
. exporting database links
. exporting sequence numbers
. exporting cluster definitions
. about to export SERVICE_NAME's tables via Conventional Path ...
. exporting synonyms
. exporting views
. exporting stored procedures
. exporting operators
. exporting referential integrity constraints
. exporting triggers
. exporting indextypes
. exporting bitmap, functional and extensible indexes
. exporting posttables actions
. exporting snapshots
. exporting snapshot logs
. exporting job queues
. exporting refresh groups and children
. exporting dimensions
. exporting post-schema procedural objects and actions
. exporting statistics
Export terminated successfully without warnings.If you don't have any warnings, proceed to automate the backups.
Automating backups is accomplished using the UNIX crontab facility.
While still root, run the following command. You can replace the EDITOR="emacs -nw" portion with whatever editor your prefer, such as EDITOR=vi.
root:~# export EDITOR="emacs -nw"
root:~# crontab -eNow add the following line on a line by itself
0 23 * * * /usr/sbin/export-oracleSave the file, exit the editor. Verify that the addition succeeded by checking the output of the following command.
root:~# crontab -l | grep export-oracle
0 23 * * * /usr/sbin/export-oracle
root:~# exit
; LogoutIf you see the line, go ahead and log out.
Set up nightly Postgres exports
(This is not required for the Reference install.) Dowload this script to /tmp. At the top of the script are several variables that you'll need to customize:
bak - location where you want local backups to be saved
servername - name of your server (and database instance)
ftp_user - username on your ftp account
ftp_password - password on your ftp account
ftp_dir - path on the remote server where your backups will be uploaded
ftp_server - your ftp server
Next, we'll save this file to our server's tcl directory so that it will be loaded on startup. It will automatically be run every night at midnight. Note that this script only backs up the database - not the OpenACS scripts and file content.
joeuser:~$ cp /tmp/acs-pgbackup-init.txt ~/web/birdnotes/tcl/acs-pgbackup-init.tcl
joeuser:~$ restart-aolserver birdnotesThat's it! The script will email you with each successful backup (or if it fails, it will send you an email with the reason)
($Id: backup-recovery.html,v 1.1.2.8 2003/05/07 17:40:58 donb Exp $)
The purpose of backup is to enable recovery. Backup and recovery are always risky; here are some steps that minimize the chance recovery is necessary:
Store everything on a fault-tolerant disk array (RAID 1 or 5 or better).
Use battery backup.
Use more reliable hardware, such as SCSI instead of IDE.
These steps improve the chances of successful recovery:
Store backups on a third disk on another controller
Store backups on a different computer on a different network in a different physical location. (Compared to off-line backup such as tapes and CDRs, on-line backup is faster and more likely to succeed, but requires maintenance of another machine.)
Plan and configure for recovery from the beginning.
Test your recovery strategy from time to time.
Make it easy to maintain and test your recovery strategy, so that you are more likely to do it.
OpenACS installations comprise files and database contents. If you follow the reference install and put all files, including configuration files, in /web/servicename/, and back up the database nightly to a file in /web/servicename/database-backup, then you can apply standard file-based backup strategies to the root directory.
Set up Nightly Postgres Exports
A quick way to automate database backup is a cron job. (This should moved into OpenACS's scheduled task project so that it's integrated with OpenACS's alerts and such.)
[service0@yourserver service0]$ export EDITOR=emacs;crontab -eAdd this line to the file. The numbers and stars at the beginning are cron columns that specify when the program should be run - in this case, whenever the minute is 0 and the hour is 1, i.e., 1:00 am every day.
0 1 * * * /usr/local/pgsql/bin/pg_dump -f /web/service0/database-backup/service0_$(date +%Y-%m-%d).dmp service0Back up the file system
Here's a quick manual way to back up a reference install - it should be replaced by an automated script within OpenACS. The command excludes the auto-generated supervise directory, which is unneccesary and has complicated permissions. Make sure that you are using the cron job to back up the database to a file in /web/service0/database-backup so that the tar command will include the database.
[root@yourserver root]# su - service0
[service0@yourserver service0]$ tar -cpsj --exclude /web/service0/etc/daemontools/supervise --file /tmp/service0-backup.tar.bz2 /web/service0/
tar: Removing leading `/' from member names
[service0@yourserver service0]$Testing
On a test service, make sure that your backup-recovery process work. After backing up the database and file system, delete the service as detailed below and then recover it.
[root@yourserver root]# svc -d /service/service0
[root@yourserver root]# mv /web/service0/ /web/service0.lost
[root@yourserver root]# rm /service/service0
rm: remove symbolic link `/service/service0'? y
[root@yourserver root]# ps -auxw | grep service0
root 1496 0.0 0.0 1312 252 ? S 16:58 0:00 supervise service0
[root@yourserver root]# kill 1496
[root@yourserver root]# ps -auxw | grep service0
[root@yourserver root]# su - postgres
[postgres@yourserver pgsql]$ dropdb service0
DROP DATABASE
[postgres@yourserver pgsql]$ dropuser service0
DROP USER
[postgres@yourserver pgsql]$ exit
logout
[root@yourserver root]#Recovery
Restore the operating system and required software. You can do this with standard backup processes or by keeping copies of the install material (OS CDs, OpenACS tarball and supporting software) and repeating the install guide.
Restore the OpenACS service. Assuming the user already exists, restore the database and files from backup and restore the daemontools link. (Because of a bug in Postgres backup-recovery, not all database objects are created in the correct order. To compensate, pre-creating some objects usually work.)
[root@yourserver root]# su - postgres
[postgres@yourserver pgsql]$ createuser service0
Shall the new user be allowed to create databases? (y/n) y
Shall the new user be allowed to create more new users? (y/n) y
CREATE USER
[postgres@yourserver pgsql]$ exit
logout
[root@yourserver root]# su - service0
[service0@yourserver service0]$ cd /web
[service0@yourserver web]$ tar xjf /tmp/service0-backup.tar.bz2
[service0@yourserver web]$ chmod -R 700 service0
[service0@yourserver web]$ createdb service0
CREATE DATABASE
[service0@yourserver web]$ psql -f /web/service0/packages/acs-kernel/sql/postgresql/postgresql.sql service0
(many lines omitted)
[service0@yourserver web]$ psql service0 < /web/service0/database-backup/database-backup.dmp
(many lines omitted)
[service0@yourserver web]$ exit
[root@yourserver root]# ln -s /web/service0/etc/daemontools /service/service0
[root@yourserver root]# sleep 10
[root@yourserver root]# svgroup web /service/service0
[root@yourserver root]#Other Backup Strategies
Earlier strategies, included here because this section hasn't been fully updated yet.
Set Up Nightly Oracle Exports
(This has not yet been updated to fit with the Reference install. To do so, edit the backup script to save the backup file in /web/servicename/database-backup). While you're working with Oracle, you should configure it to do automatic exports. An export is a separate backup copy of the database. This copy includes all of the database's state at the time that the export was initiated. If your database is corrupted, you can restore from one of these backups. You should do this step as root.
Download the backup script. Save the file export-oracle.txt as /tmp/export-oracle.txt
Login as root. The following commands will install the export script:
joeuser:~$ su -
Password: ***********
root:~# cp /tmp/export-oracle.txt /usr/sbin/export-oracle
root:~# chmod 700 /usr/sbin/export-oracleSetup the export directory; this is the directory where backups will be stored. We recommend the directory /ora8/m02/oracle-exports.
root:~# mkdir /ora8/m02/oracle-exports
root:~# chown oracle.dba /ora8/m02/oracle-exports
root:~# chmod 770 /ora8/m02/oracle-exportsNow edit /usr/sbin/export-oracle and change the SERVICE_NAME and DATABASE_PASSWORD fields to their correct values. If you want to use a directory other than /ora8/m02/oracle-exports, you also need to change the exportdir setting.
Test the export procedure by running the command:
root:~# /usr/sbin/export-oracle
mv: /ora8/m02/oracle-exports/oraexport-service_name.dmp.gz: No such file or directory
Export: Release 8.1.6.1.0 - Production on Sun Jun 11 18:07:45 2000
(c) Copyright 1999 Oracle Corporation. All rights reserved.
Connected to: Oracle8i Enterprise Edition Release 8.1.6.1.0 - Production
With the Partitioning option
JServer Release 8.1.6.0.0 - Production
Export done in US7ASCII character set and US7ASCII NCHAR character set
. exporting pre-schema procedural objects and actions
. exporting foreign function library names for user SERVICE_NAME
. exporting object type definitions for user SERVICE_NAME
About to export SERVICE_NAME's objects ...
. exporting database links
. exporting sequence numbers
. exporting cluster definitions
. about to export SERVICE_NAME's tables via Conventional Path ...
. exporting synonyms
. exporting views
. exporting stored procedures
. exporting operators
. exporting referential integrity constraints
. exporting triggers
. exporting indextypes
. exporting bitmap, functional and extensible indexes
. exporting posttables actions
. exporting snapshots
. exporting snapshot logs
. exporting job queues
. exporting refresh groups and children
. exporting dimensions
. exporting post-schema procedural objects and actions
. exporting statistics
Export terminated successfully without warnings.If you don't have any warnings, proceed to automate the backups.
Automating backups is accomplished using the UNIX crontab facility.
While still root, run the following command. You can replace the EDITOR="emacs -nw" portion with whatever editor your prefer, such as EDITOR=vi.
root:~# export EDITOR="emacs -nw"
root:~# crontab -eNow add the following line on a line by itself
0 23 * * * /usr/sbin/export-oracleSave the file, exit the editor. Verify that the addition succeeded by checking the output of the following command.
root:~# crontab -l | grep export-oracle
0 23 * * * /usr/sbin/export-oracle
root:~# exit
; LogoutIf you see the line, go ahead and log out.
Set up nightly Postgres exports
(This is not required for the Reference install.) Dowload this script to /tmp. At the top of the script are several variables that you'll need to customize:
bak - location where you want local backups to be saved
servername - name of your server (and database instance)
ftp_user - username on your ftp account
ftp_password - password on your ftp account
ftp_dir - path on the remote server where your backups will be uploaded
ftp_server - your ftp server
Next, we'll save this file to our server's tcl directory so that it will be loaded on startup. It will automatically be run every night at midnight. Note that this script only backs up the database - not the OpenACS scripts and file content.
joeuser:~$ cp /tmp/acs-pgbackup-init.txt ~/web/birdnotes/tcl/acs-pgbackup-init.tcl
joeuser:~$ restart-aolserver birdnotesThat's it! The script will email you with each successful backup (or if it fails, it will send you an email with the reason)
($Id: backup-recovery.html,v 1.1.2.8 2003/05/07 17:40:58 donb Exp $)
Subscribe to:
Posts (Atom)
