This is a HowTo and Tutorials for Linux CentOS 5.x for server configuration setup.
Suggestions, questions, comments welcome...
Full post / more »»
Feb 8, 2021
VSFTPD Virtual users configuration (with MySQL) CentOS 5.x / RHEL 5 - HowTO example
vsFTPd Virtual Users configuration with MySQL, CentOS example, How To Set Up VSFTPD virtual users,
Setup Virtual Users and Directories in VSFTPd on CentOS 5.x/6.x, RHEL 5/6 (in my case it was CentOS 5.4 x86 32bit).
( based on Virtual Hosting With vsftpd And MySQL On Debian Etch)
Someone might find this useful, so you don't have to lose a day or two for getting it work... (as I did)...
Then Start mysqld if not already:
and create root password for MySQL (if not already done):
3 Create The MySQL Database For vsftpd
login to mysql:
Create database for users:
4 Configure VSFTPD (Very Secure FTP server):
Create a non-privileged user called 'vsftpd' (with the homedir /home/vsftpd) belonging to the group 'users'. Vsftpd will run with this users privileges so risk to the system is minimized and the FTP directories of our virtual users will be in the '/home/vsftpd' directory (e.g. /home/vsftpd/user1, /home/vsftpd/user2, etc.) or as defined in VSFTPD PER USER config file.
First we empty the existing file and then open it for editing:
However, create that directory now:
Now you must configure PAM (Password Authentication) so that it uses the MySQL database to authenticate your virtual FTP users instead of /etc/passwd and /etc/shadow.
The PAM configuration for vsftpd is in /etc/pam.d/vsftpd.
Make a backup of the original file and create a new one like this:
Find here (pbone.net) the RPM pam_mysql module to download it (use 'wget' is simple), at this time of writing it was 'pam_mysql-0.7-0.5.rc1.el5.kb.2.i386.rpm' (watch for the right version i386 or x86_64 if you have 64bit system)
and install it:
When installed, you should find it:
Insert users to database you can use the MySQL shell:
Now create the virtual user 'user1' with the password 'secret' (which will be stored encrypted using MySQL's MD5 function):
Now restart/start VSFTPD
if not ... I'm sorry, try read again.
How to add more users in the future when you need.. it's easy in 2 steps:
1. add new user ( e.g. 'user12' with passw 'secret12', you can use the full name with email address also if you want, like 'user12@example.com' ) :
Or you can use phpMyAdmin if you have a website running on the server (just download from phpMyAdmin site and extract to a subdir at your site - for example /var/www/mysite/phpmyadmin):
1. Login with root account (for now only 'root' has the rights to create/modify the vsFTP table at this time - you can create another user with privileges to modify the tables)
2. On the left side select 'vsftpd' database
3. then select table 'accounts'
4. On top select tab Browse once you have selected the table 'accounts'
5. you should see the list of users in the database:
6. Now to ADD new user: select INSERT tab on top
- in the field 'username' type the user's name for login (ie. 'newuser')
- in the field 'pass' select MD5 from dropdown list and type password for 'newuser' <- type it in plain text as it will be saved as MD5 because of field type selected MD5) Don't enter anything else, just click GO to save 'newuser' to database:
7. you should see the lines:
Inserted rows: 1
Inserted row id: "some number in list order automatically assigned id"
8. To see if you have added 'newuser' click again top tab Browse and the list should show the name and MD5 password for 'newuser'
Then you can repeat the step 6. as many times you need to add users.
9. To end phpMyAdmin session click Log Out / Exit :
Hopefully this is it and I'd be happy to see any comments of success (or fails).
TY for reading it ...
Full post / more »»
Setup Virtual Users and Directories in VSFTPd on CentOS 5.x/6.x, RHEL 5/6 (in my case it was CentOS 5.4 x86 32bit).
( based on Virtual Hosting With vsftpd And MySQL On Debian Etch)
Someone might find this useful, so you don't have to lose a day or two for getting it work... (as I did)...
Advantages
1. Storing users and passwords into one database is easier to maintain and you avoid having local accounts for all the users you might need to give them FTP access, so the security risk of hacking user accounts is minimized. All users are located in one directory with user specific settings if needed.
2. MySQL protects databases with user specific permissions granted by MySQL root (a superuser for databases, giving them access, permissions to read, write, modify...)
So the MySQL superuser root should have its own MySQL password (not the same as account 'root') in case of exploits to mysql and hacking the local 'root' account to get access to the server ( some more MySQL basics )
1. Storing users and passwords into one database is easier to maintain and you avoid having local accounts for all the users you might need to give them FTP access, so the security risk of hacking user accounts is minimized. All users are located in one directory with user specific settings if needed.
2. MySQL protects databases with user specific permissions granted by MySQL root (a superuser for databases, giving them access, permissions to read, write, modify...)
So the MySQL superuser root should have its own MySQL password (not the same as account 'root') in case of exploits to mysql and hacking the local 'root' account to get access to the server ( some more MySQL basics )
REQUIREMENTS:
pam_mysql.so library
You will need (if not already installed) VSFTPD and MySQL:pam_mysql.so library
yum install vsftpd mysql-server
Then Start mysqld if not already:
service mysqld restart
and create root password for MySQL (if not already done):
mysqladmin -u root password yourrootsqlpassword
3 Create The MySQL Database For vsftpd
login to mysql:
mysql -u root -penter " yourrootsqlpassword " - Be aware: yourrootsqlpassword IS NOT your user's 'root' password and should be different.
Create database for users:
CREATE DATABASE vsftpd;still in the MySQL shell, create the database table needed (there is only one table with usernames and passwords MD5 encrypted):
GRANT SELECT ON vsftpd.* TO 'vsftpd'@'localhost' IDENTIFIED BY 'vsftpdpassword';
FLUSH PRIVILEGES;
USE vsftpd;then you can
CREATE TABLE `accounts` (
`id` INT NOT NULL AUTO_INCREMENT PRIMARY KEY ,
`username` VARCHAR( 30 ) NOT NULL ,
`pass` VARCHAR( 50 ) NOT NULL ,
UNIQUE ( `username` )
) ENGINE = MYISAM ;
exit;
4 Configure VSFTPD (Very Secure FTP server):
Create a non-privileged user called 'vsftpd' (with the homedir /home/vsftpd) belonging to the group 'users'. Vsftpd will run with this users privileges so risk to the system is minimized and the FTP directories of our virtual users will be in the '/home/vsftpd' directory (e.g. /home/vsftpd/user1, /home/vsftpd/user2, etc.) or as defined in VSFTPD PER USER config file.
useradd -G users -s /sbin/nologin -d /home/vsftpd vsftpdThen make VSFTP config settings (make a backup of the original /etc/vsftpd.conf file):
cp -v /etc/vsftpd/vsftpd.conf /etc/vsftpd/vsftpd.conf-origand make our own needed changes:
First we empty the existing file and then open it for editing:
cat /dev/null > /etc/vsftpd/vsftpd.confvsftpd.conf configuration settings (copy this into file):
vi /etc/vsftpd/vsftpd.conf
# No ANONYMOUS users allowedWith the user_config_dir option you can specify a directory for per-user configuration files that override parts of the global settings. This is totally optional and up to you if you want to use this feature.
anonymous_enable=NO
# Allow 'local' users with WRITE permissions (0755)
local_enable=YES
write_enable=YES
local_umask=022
dirmessage_enable=YES
xferlog_enable=YES
# if you want to LOG vsftpd activity then uncomment this log_ftp_protocol
# log_ftp_protocol=YES
connect_from_port_20=YES
# uncomment xferlog_file and xferlog_std_format if you DIDN'T use the line above
# with log_ftp_protocol - it must be excluding each other
# The name of log file when xferlog_enable=YES and xferlog_std_format=YES
# WARNING - changing this filename affects /etc/logrotate.d/vsftpd.log
#xferlog_file=/var/log/xferlog
#
# xferlog_std_format Switches between logging into vsftpd_log_file and xferlog_file files.
# NO writes to vsftpd_log_file, YES to xferlog_file
# xferlog_std_format=YES
#
# You may change the default value for timing out an idle session (in seconds).
#idle_session_timeout=600
#
# You may change the default value for timing out a data connection (in seconds).
#data_connection_timeout=120
#
# define a unique user on your system which the
# ftp server can use as a totally isolated and unprivileged user.
nopriv_user=vsftpd
chroot_local_user=YES
listen=YES
# here we use the authentication module for vsftpd to check users name and passw
pam_service_name=vsftpd
userlist_enable=YES
tcp_wrappers=YES
# If userlist_deny=YES (default), never allow users in this file
# /etc/vsftpd/user_list , and do not even prompt for a password.
# Note that the default vsftpd pam config also checks /etc/vsftpd/ftpusers
# for users that are denied.
userlist_deny=yes
# here the vsftpd will allow the 'vsftpd' user to login into '/home/vsftpd/$USER directory
guest_enable=YES
guest_username=vsftpd
local_root=/home/vsftpd/$USER
user_sub_token=$USER
virtual_use_local_privs=YES
user_config_dir=/etc/vsftpd/vsftpd_user_conf
force_local_data_ssl=NO
force_local_logins_ssl=NO
# PASV - passive ports for FTP (range 44000 - 44100 ; 100 PASV ports,
# REMEMBER to OPEN FIREWALL FOR ALLOWING FTP Passive CONNECTIONS
# check "how to enable Passive FTP in IPTABLES": here or here
pasv_enable=YES
pasv_min_port=44000
pasv_max_port=44100
However, create that directory now:
mkdir /etc/vsftpd/vsftpd_user_confIf you want to have for example: 'user1' to have different 'home dir' other than '/home/vsftpd/user1' then create
vsftpd PER USER configuration file:
vi /etc/vsftpd/vsftpd_user_conf/user1with configuration settings in it:
dirlist_enable=YESThe 'user1' directory must be created if you want the user to be able to login!
download_enable=YES
# full path to the directory where 'user1' will have access, change to your needs
local_root=/home/users/user1
write_enable=YES
mkdir /home/users/user1and giving 'user1' the permissions to read, write...:
chmod 700 /home/users/user1So now user1 has 'home dir' in '/home/users/user1' instead of '/home/vsftpd/user1' and it can be changed to whatever you need to in the Per user configuration file ...
chown vsftpd.users /home/users/user1
Now you must configure PAM (Password Authentication) so that it uses the MySQL database to authenticate your virtual FTP users instead of /etc/passwd and /etc/shadow.
The PAM configuration for vsftpd is in /etc/pam.d/vsftpd.
Make a backup of the original file and create a new one like this:
cp /etc/pam.d/vsftpd /etc/pam.d/vsftpd-origthe /etc/pam.d/vsftpd contents (note: this should be only 4 lines when you copy it):
cat /dev/null > /etc/pam.d/vsftpd
vi /etc/pam.d/vsftpd
#%PAM-1.0AND MAKE SURE that you replace the MySQL 'vsftpdpassword' password with your own one used before in 3 Create The MySQL Database For vsftpd
session optional pam_keyinit.so force revoke
auth required pam_mysql.so user=vsftpd passwd=vsftpdpassword host=localhost db=vsftpd table=accounts usercolumn=username passwdcolumn=pass crypt=3
account required pam_mysql.so user=vsftpd passwd=vsftpdpassword host=localhost db=vsftpd table=accounts usercolumn=username passwdcolumn=pass crypt=3
Now comes that tricky part for CentOS to make it work !
You need pam_mysql.so library, which is not included in CentOS installation or is not YUM installable, so you have to install from RPM (or EPEL repository ... or whichever method you prefer).Find here (pbone.net) the RPM pam_mysql module to download it (use 'wget' is simple), at this time of writing it was 'pam_mysql-0.7-0.5.rc1.el5.kb.2.i386.rpm' (watch for the right version i386 or x86_64 if you have 64bit system)
and install it:
rpm -Uvh pam_mysql-0.7-0.5.rc1.el5.kb.2.i386.rpmIt should install without warnings or error... else ... I recommend you use search in google to make it work!
When installed, you should find it:
ls -al /lib/security/pam_m*there it is in the last line in this example ! (you can have more, but should be in there)
-rwxr-xr-x 1 root root 8024 Sep 4 00:51 /lib/security/pam_mail.so
-rwxr-xr-x 1 root root 15848 Sep 4 00:51 /lib/security/pam_mkhomedir.so
-rwxr-xr-x 1 root root 3892 Sep 4 00:51 /lib/security/pam_motd.so
-rwxr-xr-x 1 root root 36920 Feb 28 2008 /lib/security/pam_mysql.so
This is critical for use virtual users auth with MySQL database
Now 5 Create The First Virtual UserInsert users to database you can use the MySQL shell:
mysql -u root -penter password ...
USE vsftpd;use the database 'vsftpd'
Now create the virtual user 'user1' with the password 'secret' (which will be stored encrypted using MySQL's MD5 function):
INSERT INTO accounts (username, pass) VALUES('user1', md5('secret'));You should now have one user in database:
mysql> select * from accounts;Now user1's homedir is '/home/vsftpd/user1' , unfortunately vsftpd doesn't create that directory automatically if it doesn't exist. Therefore create it manually now and make it owned by the vsftpd user and group 'users':
+----+-----------+----------------------------------+
| id | username | pass |
+----+-----------+----------------------------------+
| 1 | user1 | 5ebe2294ecd0e0f08eab7690d2a6ee69 |
+----+-----------+----------------------------------+
1 rows in set (0.00 sec)
exit;
mkdir /home/vsftpd/user1
chown vsftpd:users /home/vsftpd/user1
Now restart/start VSFTPD
service vsftpd restartand you should probably be able to login to your FTP server with some of the Windows clients like WS_FTP or SmartFTP or whatever you like...
if not ... I'm sorry, try read again.
How to add more users in the future when you need.. it's easy in 2 steps:
1. add new user ( e.g. 'user12' with passw 'secret12', you can use the full name with email address also if you want, like 'user12@example.com' ) :
mysql -u root -p2. make new 'user12' home dir
USE vsftpd;
INSERT INTO accounts (username, pass) VALUES('user12', md5('secret12'));
exit;
mkdir /home/vsftpd/user12
chown vsftpd:users /home/vsftpd/user12
Or you can use phpMyAdmin if you have a website running on the server (just download from phpMyAdmin site and extract to a subdir at your site - for example /var/www/mysite/phpmyadmin):
1. Login with root account (for now only 'root' has the rights to create/modify the vsFTP table at this time - you can create another user with privileges to modify the tables)
2. On the left side select 'vsftpd' database
3. then select table 'accounts'
4. On top select tab Browse once you have selected the table 'accounts'
5. you should see the list of users in the database:
6. Now to ADD new user: select INSERT tab on top
- in the field 'username' type the user's name for login (ie. 'newuser')
- in the field 'pass' select MD5 from dropdown list and type password for 'newuser' <- type it in plain text as it will be saved as MD5 because of field type selected MD5) Don't enter anything else, just click GO to save 'newuser' to database:
7. you should see the lines:
Inserted rows: 1
Inserted row id: "some number in list order automatically assigned id"
8. To see if you have added 'newuser' click again top tab Browse and the list should show the name and MD5 password for 'newuser'
Then you can repeat the step 6. as many times you need to add users.
9. To end phpMyAdmin session click Log Out / Exit :
Hopefully this is it and I'd be happy to see any comments of success (or fails).
TY for reading it ...
Full post / more »»
Feb 6, 2020
Incremental Backup with TAR / simple FTP to another location and email status
This is my script for archiving incremental and full backup with TAR in Linux and then FTP that archive to another 'server' for security if server loses data.
This is not complete backup to make it possible to restore 'bare metal' for that case use Mondo Backup.
After I spent some time discovering The BIG BANG of Universe and The Meaning of Life :lol I managed somehow to create a script to make some backup of files on server and TAR/GZIP it and then FTP the archive to another FTP server and finally email the results.
This script also measures time needed to complete it and deletes archive older than xx days (set in find -mtime +20) and makes incremental backup every weekday and then FULL BACKUP on Sundays (which suits me bcoz no heavy load).
Put the scripts (files) to some directory where you will be making backups to, I use
Files for TAR to include and exclude are in plain txt format and filenames listed each name in separate line (these paths will be included in TAR-GZIP archive):
file: including.txt:
For excluding the files / directories the syntax is:
- var/tmp <-- exclude directory matching 'var/tmp' in the name
- spool <-- exclude files matching 'spool' in the name: e.g. 'spool1 spoolwhatever' also *_log* matches names including '_log'
- var/tmp/serverbackups <-- exclude directory with backups in it so we don't archive ourselves when creating new archives - obviously !
file: excluding.txt:
I'm using LFTP to make sure that FTP transaction runs complete since ftp didn't always finish, the script for transferring the BACKUP file:
lftpscript.sh:
Crontab running the script, this is located in PATH '/usr/local/bin':
Then it runs the backup script.
file: archive.sh:
Now comes the 'real' script which handles the archiving with TAR and sends over FTP to another location.
file: backup.sh:
This should do it and you should have a copy of file transferred over FTP to other server.
Otherwise I would use Duplicity for backing up the data, but it's a little more complicated to configure 'include and exclude' ...
Full post / more »»
This is not complete backup to make it possible to restore 'bare metal' for that case use Mondo Backup.
REQUIREMENTS:
The script requires GNU TAR which is capable of Incremental archiving
Optional:
The LFTP improved FTP client for Linux capable of auto retrying the transfer until finished successfully
The script requires GNU TAR which is capable of Incremental archiving
Optional:
The LFTP improved FTP client for Linux capable of auto retrying the transfer until finished successfully
After I spent some time discovering The BIG BANG of Universe and The Meaning of Life :lol I managed somehow to create a script to make some backup of files on server and TAR/GZIP it and then FTP the archive to another FTP server and finally email the results.
This script also measures time needed to complete it and deletes archive older than xx days (set in find -mtime +20) and makes incremental backup every weekday and then FULL BACKUP on Sundays (which suits me bcoz no heavy load).
This is the script I had written to work for ME (you will have to modify it for yourself, I hope you find what and where), since I put it in CRON making it run every day
Put the scripts (files) to some directory where you will be making backups to, I use
/usr/tmp/serverbackups
Files for TAR to include and exclude are in plain txt format and filenames listed each name in separate line (these paths will be included in TAR-GZIP archive):
file: including.txt:
/var/
/etc/
/home/
For excluding the files / directories the syntax is:
- var/tmp <-- exclude directory matching 'var/tmp' in the name
- spool <-- exclude files matching 'spool' in the name: e.g. 'spool1 spoolwhatever' also *_log* matches names including '_log'
- var/tmp/serverbackups <-- exclude directory with backups in it so we don't archive ourselves when creating new archives - obviously !
file: excluding.txt:
var/tmp/serverbackups
proc
*_log*
var/tmp
var/lib/bluetooth
var/lib/cs
var/lib/dav
var/lib/dbus
var/lib/dhcpv6
var/lib/dovecot
var/lib/games
var/lib/rpm
var/lib/webalizer
var/lib/yum
var/log
var/run
var/www/manual
var/yp
var/lib/php/session
spool
var/cache
*zip
*gz
etc/rc*
home/httpd/manual
rpm
I'm using LFTP to make sure that FTP transaction runs complete since ftp didn't always finish, the script for transferring the BACKUP file:
lftpscript.sh:
#!/bin/sh
HOST='ftp.domain.com'
USER='ftpuser'
PASSWD='idontknow'
FILE=$(cat archivename.txt)
lftp -c "open $HOST && user $USER $PASSWD && cd FOLDER_NAME_FOR_STORING/backups/ && put $FILE" <<END_SCRIPT
# all in one command that connects to HOST=ftp.domain.com with Username/password
# and changes directory to whatever you need
# then transfers the file with the name of created archive in archivename.txt (ex. BACKUP-2009-12-15-Fri-01-15-01h.tgz)
bye
exit
END_SCRIPT
exit 0
Crontab running the script, this is located in PATH '/usr/local/bin':
PATH=/sbin:/bin:/usr/bin:/usr/sbin:/usr/local/binruns the script archive.sh which only changes the directory path to '/usr/tmp/serverbackups' where the real Backup script is located.
0 1 * * * archive.sh >/dev/null # runs the script at 1.00AM every day
Then it runs the backup script.
file: archive.sh:
#!/bin/bash
cd /usr/tmp/server_backup
./backup.sh # the real script to make backup
Now comes the 'real' script which handles the archiving with TAR and sends over FTP to another location.
file: backup.sh:
#!/bin/sh
# DELETE archive older than -mtime +'days'
find . -name 'BACKUP*.tgz' -mtime +20 -delete
find . -name 'stopwatch*' -mtime +2 -delete
start1=$(date +"%T h ( %s )")
start=$(date +%s)
# on SUNDAY make FULL backup
if [ $(date +"%a") = "Sun" ]; then
{
SNAPSHOTFILE="./usr-full"; # needed by TAR (GNU-TAR to be precise) which is used to compare for incremental backups
ARCHIVENAME=BACKUP-full-$(date +"%F-%a-%H-%M-%Sh").tgz; # self explaining
rm -f "./usr-full";
}
else
{
ARCHIVENAME=BACKUP-$(date +"%F-%a-%H-%M-%Sh").tgz;
# a name of a backup file: BACKUP-2009-12-15-Fri-01-15-01h.tgz
SNAPSHOTFILE="./usr-1";
cp "./usr-full" "./usr-1";
}
fi
echo $ARCHIVENAME >archivename.txt # need the name to FTP transfer so store it in file archivename.txt which doesn't change (used later in lftpscript.sh ) !
# creating text to send in email
echo "-----------------------" >stopwatch-$archivename.txt
echo "Backup of $ARCHIVENAME" >>stopwatch-$archivename.txt
echo "-----------------------" >>stopwatch-$archivename.txt
echo " " >>stopwatch-$archivename.txt # echo " " makes new line /CR or LF whatever it does
# I do not need this precise time { time tar -T including.txt -X excluding.txt -pczvRf $ARCHIVENAME; } 2>> stopwatch-$ARCHIVENAME.txt >/dev/null
{ tar -T including.txt -X excluding.txt -pczvR --listed-incremental=$SNAPSHOTFILE -f $ARCHIVENAME; } 2>> stopwatch-$ARCHIVENAME.txt >/dev/null
stopped1=$(date +"%T h ( %s )")
stopped=$(date +%s)
ftpstarted=$stopped
thetime=$(($stopped-$start)) # doing some math in shell that's why $()
ELAPSEDHRS=$((($thetime%86400/3600)))
ELAPSSEC=$(($thetime%3600))
echo " " >>stopwatch-$ARCHIVENAME.txt
echo -n "File Size (Byte-s) : " >>stopwatch-$ARCHIVENAME.txt
ls -al "$ARCHIVENAME" | cut -f 5 -d ' ' >>stopwatch-$ARCHIVENAME.txt
# this part | cut -f 5 -d ' ' is sometimes maybe 6 instead of 5, experiment which gives you only the SIZE of the file
echo " " >>stopwatch-$ARCHIVENAME.txt
echo "Started: " $(date -d "1970-01-01 $start sec UTC" +"%A, %d.%m.%Y, %H:%M:%S") >>stopwatch-$ARCHIVENAME.txt #outputs: Sunday, 05.08.2012, 07:16:17
echo "Stopped: " $(date -d "1970-01-01 $stopped sec UTC" +"%A, %d.%m.%Y, %H:%M:%S") >>stopwatch-$ARCHIVENAME.txt
echo "Time needed: " $(($thetime/86400))" days, "$ELAPSEDHRS" hours, "$(($ELAPSSEC/60))" minutes, "$(($ELAPSSEC%60))" seconds ( $thetime secs)" >>stopwatch-$ARCHIVENAME.txt
# outputs: Time needed: 0 days, 0 hrs, 3 minutes, 14 seconds / 194 secs
# outputs: Time needed: 28 days, 2 hrs, 22 minutes, 53 seconds / 2427773 secs
echo "-----------------------" >>stopwatch-$ARCHIVENAME.txt
echo " " >>stopwatch-$ARCHIVENAME.txt
echo "FTP start:" >>stopwatch-$ARCHIVENAME.txt
# again I dont need exact time procedure { time ./lftpscript.sh; } 2>> stopwatch-$ARCHIVENAME.txt
{ ./lftpscript.sh; } 2>> stopwatch-$ARCHIVENAME.txt
ftpstop1=$(date +"%T h ( %s )")
ftpstopped=$(date +%s)
ftptime=$(($ftpstopped-$ftpstarted))
FTPELAPSEDHRS=$((($ftptime%86400/3600)))
FTPELAPSSEC=$(($ftptime%3600))
echo " " >>stopwatch-$ARCHIVENAME.txt
echo "Start of FTP: " $(date -d "1970-01-01 $stopped sec UTC" +"%A, %d.%m.%Y, %H:%M:%S") >>stopwatch-$ARCHIVENAME.txt
echo "End of FTP: " $(date -d "1970-01-01 $ftpstopped sec UTC" +"%A, %d.%m.%Y, %H:%M:%S") >>stopwatch-$ARCHIVENAME.txt
echo "Time of FTP transfer: " $(($ftptime/86400))" days, "$FTPELAPSEDHRS" hours, "$(($FTPELAPSSEC/60))" minutes, "$(($FTPELAPSSEC%60))" seconds ( $ftptime secs)" >>stopwatch-$ARCHIVENAME.txt
mail -s "Backup of $ARCHIVENAME" "email address of recipient" <stopwatch-$ARCHIVENAME.txt
#finally email report :-)
This should do it and you should have a copy of file transferred over FTP to other server.
Otherwise I would use Duplicity for backing up the data, but it's a little more complicated to configure 'include and exclude' ...
Full post / more »»
Oct 1, 2011
Adaptec ARCCONF getconfig - check Adaptec RAID array status
/usr/Storman/arcconf getconfig 1 al - Lists information about the controllers, logical drives, and physical devices.
To check the health of Adaptec RAID array on CentOS 5 server (RHEL 5 based, also Fedora) I have modified this script using ARCCONF and run it in CRON to get emailed about the status of RAID array (I have Adaptec RAID 3405 controller).
Adaptec Storage Manager software for managing, monitoring and checking the Adaptec RAID arrays is unlike 3Ware 3DM2 manager, not willing to run as a background service on CentOS / RHEL 5 based system.
When Adaptec utilities are installed, you can get your RAID array information with the command:
You should examine the output of the arcconf command on your system before you use the script and edit it if necessary.
Now that's all ok, but if something goes bad you will not know about it until you check it again manually.
This made me do the script to check from CRON (# crontab -l -- view cron, # crontab -e -- edit cron) every hour and email me the status if something wrong (or just a status report on Wednesday and Saturday - you can modify it when you want)
arctest_status.sh
Now that's what I wanted !
and on Wednesday/Saturday I get an email with status check like this:
Full post / more »»
To check the health of Adaptec RAID array on CentOS 5 server (RHEL 5 based, also Fedora) I have modified this script using ARCCONF and run it in CRON to get emailed about the status of RAID array (I have Adaptec RAID 3405 controller).
Adaptec Storage Manager software for managing, monitoring and checking the Adaptec RAID arrays is unlike 3Ware 3DM2 manager, not willing to run as a background service on CentOS / RHEL 5 based system.
You will need to download and install Adaptec utilites
and something else what I'm using is a great e-mailer script sendEmail which enables me to send email with attachment from CLI.
and something else what I'm using is a great e-mailer script sendEmail which enables me to send email with attachment from CLI.
When Adaptec utilities are installed, you can get your RAID array information with the command:
# /usr/StorMan/arcconf getconfig 1 althe output should be like this:
Controllers found: 1 ---------------------------------------------------------------------- Controller information ---------------------------------------------------------------------- Controller Status : Optimal Channel description : SAS/SATA Controller Model : Adaptec 3405 Controller Serial Number : 7C2110BD455 Physical Slot : 3 Temperature : 49 C/ 120 F (Normal) Installed memory : 128 MB Copyback : Disabled Background consistency check : Disabled Automatic Failover : Enabled Defunct disk drive count : 0 Logical devices/Failed/Degraded : 1/0/0 -------------------------------------------------------- Controller Version Information -------------------------------------------------------- BIOS : 5.2-0 (15753) Firmware : 5.2-0 (15753) Driver : 1.1-5 (2453) Boot Flash : 5.2-0 (15753) -------------------------------------------------------- Controller Battery Information -------------------------------------------------------- Status : Optimal Over temperature : No Capacity remaining : 99 percent Time remaining (at current draw) : 3 days, 0 hours, 52 minutes ---------------------------------------------------------------------- Logical device information ---------------------------------------------------------------------- Logical device number 0 Logical device name : RAID10 RAID level : 10 Status of logical device : Optimal Size : 279800 MB Stripe-unit size : 256 KB Read-cache mode : Enabled Write-cache mode : Enabled (write-back) Write-cache setting : Enabled (write-back) when protected by battery Partitioned : Yes Protected by Hot-Spare : No Bootable : Yes Failed stripes : No -------------------------------------------------------- Logical device segment information -------------------------------------------------------- Group 0, Segment 0 : Present (0,0) 3LN3BY8Q00009823KDMV Group 0, Segment 1 : Present (0,1) 3LN3V6AQ00009829MMLC Group 1, Segment 0 : Present (0,2) 3LN1AYYD00009747RGSB Group 1, Segment 1 : Present (0,3) 3LN2GAEC00009813AQW6 ---------------------------------------------------------------------- Physical Device information ---------------------------------------------------------------------- Device #0 Device is a Hard drive State : Online Supported : Yes Transfer Speed : SAS 3.0 Gb/s Reported Channel,Device : 0,0 Reported Location : Enclosure 0, Slot 0 Reported ESD : 2,0 Vendor : SEAGATE Model : ST3146855SS Firmware : 0002 Serial number : 3LN3BY8Q00009823KDMV World-wide name : 5000C50007BCFA20 Size : 140014 MB Write Cache : Enabled (write-back) FRU : None S.M.A.R.T. : No Device #1 Device is a Hard drive State : Online Supported : Yes Transfer Speed : SAS 3.0 Gb/s Reported Channel,Device : 0,1 Reported Location : Enclosure 0, Slot 1 Reported ESD : 2,0 Vendor : SEAGATE Model : ST3146855SS Firmware : 0002 Serial number : 3LN3V6AQ00009829MMLC World-wide name : 5000C50002F017B8 Size : 140014 MB Write Cache : Enabled (write-back) FRU : None S.M.A.R.T. : No Device #2 Device is a Hard drive State : Online Supported : Yes Transfer Speed : SAS 3.0 Gb/s Reported Channel,Device : 0,2 Reported Location : Enclosure 0, Slot 2 Reported ESD : 2,0 Vendor : SEAGATE Model : ST3146855SS Firmware : 0002 Serial number : 3LN1AYYD00009747RGSB World-wide name : 5000C50005020B14 Size : 140014 MB Write Cache : Enabled (write-back) FRU : None S.M.A.R.T. : No Device #3 Device is a Hard drive State : Online Supported : Yes Transfer Speed : SAS 3.0 Gb/s Reported Channel,Device : 0,3 Reported Location : Enclosure 0, Slot 3 Reported ESD : 2,0 Vendor : SEAGATE Model : ST3146855SS Firmware : 0002 Serial number : 3LN2GAEC00009813AQW6 World-wide name : 5000C50007BD43C0 Size : 140014 MB Write Cache : Enabled (write-back) FRU : None S.M.A.R.T. : No Device #4 Device is an Enclosure services device Reported Channel,Device : 2,0 Enclosure ID : 0 Type : SES2 Vendor : ADAPTEC Model : Virtual SGPIO 0 Firmware : 0001 Status of Enclosure services device Temperature : Normal Command completed successfully.
Now that's all ok, but if something goes bad you will not know about it until you check it again manually.
This made me do the script to check from CRON (# crontab -l -- view cron, # crontab -e -- edit cron) every hour and email me the status if something wrong (or just a status report on Wednesday and Saturday - you can modify it when you want)
arctest_status.sh
#!/bin/sh
DATE=$(date +"%F (%H:%M:%Sh)")
RAID=/var/tmp/adaptec/adaptec3405check_$(date +"%F_%H-%M-%Sh").txt
RAIDSTATUSFILE=/var/tmp/adaptec/adaptec3405status.txt
/usr/StorMan/arcconf getconfig 1 al > $RAID
CTRLSTAT=$(grep 'Controller Status' $RAID| cut -d\: -f2 | cut -d' ' -f2)
## Optimal
echo "Adaptec Status $DATE :" >$RAIDSTATUSFILE
echo "----------------------------------------" >>$RAIDSTATUSFILE
echo "Controller status : $CTRLSTAT" >>$RAIDSTATUSFILE
## CTRLBATINFO=$(grep -A 2 'Controller Battery' $RAID|grep 'Status'| cut -d\: -f2)
CTRTEMP=$(grep 'Temperature' $RAID| awk '{print $7}' | sed -e 's/^.*(\(.*\)),*/\1/')
CTRTEMPERATURE=$(grep 'Temperature' $RAID) >>$RAIDSTATUSFILE
## Normal
echo $CTRTEMPERATURE >>$RAIDSTATUSFILE
LOGICSTAT=$(grep 'Status of logical device' $RAID| cut -d\: -f2 | cut -d' ' -f2)
## Optimal
echo "Status of logical device : $LOGICSTAT" >>$RAIDSTATUSFILE
LOGICSTR=$(grep 'Failed stripes' $RAID| cut -d\: -f2 | cut -d' ' -f2)
## No
echo "Failed stripes : $LOGICSTR" >>$RAIDSTATUSFILE
# number of drives
DRIVESNO=$(grep -B 1 -A 1 'Device is a Hard' $RAID | grep -c 'Device #')
echo "Devices found : $DRIVESNO" >>$RAIDSTATUSFILE
if [ "$CTRLSTAT" = "Optimal" ]
then
# when everything is OK send the status message on Wednesday and Saturday (Wed / Sat) on 02.00 hrs, which is set to run in CRON every hour (15 * * * * /usr/local/bin/arctest_status.sh >/dev/null )
# if you don't want to get emails if nothing wrong then don't use this block if ... fi
# this should be all in 1 line
if ( [ "$(date +"%H")" = "02" ] && [ "$(date +"%a")" = "Wed" ] ) || ( [ "$(date +"%H")" = "02" ] && [ "$(date +"%a")" = "Sat" ] )
then
i="0"
while [ $i -lt "$DRIVESNO" ]
do
CURDRIVE=DRIVE$i
# this should be all in 1 line
echo "$CURDRIVE : $(grep -A 2 "Device #$i" $RAID | grep 'State' | cut -d\: -f2 | cut -d' ' -f2)" >>$RAIDSTATUSFILE
i=$[$i+1]
done
# this should be all in 1 line
/usr/local/bin/sendEmail -f "adaptec@example.com" -t "youremail@example.com" -u "Adaptec RAID status $DATE " -o message-file=$RAIDSTATUSFILE >/dev/null
fi
$(rm $RAID)
elif [ "$CTRLSTAT" != "Optimal" ]
then
## SENDTHEMAIL
cat $RAID >>$RAIDSTATUSFILE
# this should be all in 1 line
/usr/local/bin/sendEmail -f "adaptec@example.com" -t "youremail@example.com" -u "RAID FAILURE - Adaptec RAID error $DATE !" -o message-file=$RAIDSTATUSFILE -a $RAID >/dev/null
else
cat $RAID >>$RAIDSTATUSFILE
# this should be all in 1 line
/usr/local/bin/sendEmail -f "adaptec@example.com" -t "youremail@example.com" -cc "another@example.com" -u "RAID FAILURE - Adaptec RAID error $DATE !" -o message-file=$RAIDSTATUSFILE -a $RAID >/dev/null
fi
Now that's what I wanted !
and on Wednesday/Saturday I get an email with status check like this:
Adaptec Status 2011-10-01 (02:20:01h) : ---------------------------------------- Controller status : Optimal Temperature : 51 C/ 123 F (Normal) Status of logical device : Optimal Failed stripes : No Devices found : 4 DRIVE0 : Online DRIVE1 : Online DRIVE2 : Online DRIVE3 : Online
Full post / more »»
Jun 6, 2011
Linux stopwatch / get the program execution time in shell
Simple time/ stopwatch of program execution in the shell (presuming the clock on the server runs synced to NTP so it's accurate, otherwise it gets inaccurate after longer execution period):
create a shell script (ie. ftpscript.sh) and use the following code (modify the line with execute shell command):
create a shell script (ie. ftpscript.sh) and use the following code (modify the line with execute shell command):
#!/bin/shFull post / more »»
start=$(date +%s) ## get current time/date to $start
{ ./lftpscript.sh; } ## execute shell command / program ./lftpscript.sh in this example
stopped=$(date +%s)
thetime=$(($stopped-$start)) ## doing some math in shell calculating time difference
echo "Time needed: " $(date -d "1970-01-01 $thetime sec" +"%H:%M:%S") / $thetime "secs" ## prints out the time ( like: 00:01:29 / 89 secs )
Apr 21, 2010
What version of linux is running (determine installed version) ?
Determine the Installed Version of Redhat-based Systems:
you will see something like:
For kernel version, go to a terminal and run:
cat /etc/redhat-releaseor
cat /etc/issue
you will see something like:
Linux version 2.4.21-15.EL (bhcompile@bugs.build.redhat.com) (gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-34)) #1 Thu Apr 22 00:27:41 EDT 2004
ls /etc/*release*will give you a file that contains the version. Then use cat 'filename_found_ls'
ls /etc/*version*
For kernel version, go to a terminal and run:
uname -aFull post / more »»
Apr 3, 2010
VsFTPd user with global FTP privileges (like 'root')
Give VsFTPd user a global (like 'root') privileges
There's not actually need to give a Linux user FTP access to entire server; each user can access its own directory (is chrooted), which locks the user to its directory and can not browse anywhere else.
This can be useful for an admin to create a generic FTP user, with a different username and password from any users with sudo to root capabilities.
Edit vsftpd.conf:
uncomment these lines (press i to enter insert/edit mode):
and save/close the file (ESC, :, x)
Then edit the list,
and type in the user or users (one per line) which you want to break out of the chroot jail and give global FTP to.
Don't forget to restart vsftpd:
There's not actually need to give a Linux user FTP access to entire server; each user can access its own directory (is chrooted), which locks the user to its directory and can not browse anywhere else.
This can be useful for an admin to create a generic FTP user, with a different username and password from any users with sudo to root capabilities.
Edit vsftpd.conf:
cd etc/vsftpd
vim vsftpd.conf
uncomment these lines (press i to enter insert/edit mode):
chroot_list_enable=YES
chroot_list_file=/etc/vsftpd.chroot_list (this file can be anything you like, just remember how you name it !)
and save/close the file (ESC, :, x)
Then edit the list,
vim /etc/vsftpd.chroot_list
and type in the user or users (one per line) which you want to break out of the chroot jail and give global FTP to.
Don't forget to restart vsftpd:
service vsftpd restartFull post / more »»
Subscribe to:
Posts (Atom)