Wednesday, November 25, 2015

Install Postgresql 9.2 on Ubuntu 12.04 LTS

Some of the Servers I maintain are in Ubuntu 12.04 LTS and sometimes I misplace resources I need to maintain. So I keep here one of the important piece.

First sure about the language setup with

locale -a

Then us the following commands:

$ sudo apt-get update
$ sudo apt-get -y install python-software-properties
$ sudo add-apt-repository ppa:pitti/postgresql
$ sudo apt-get update

$ sudo apt-get -y install postgresql-9.2 postgresql-client-9.2 postgresql-contrib-9.2
$ sudo apt-get -y install postgresql-server-dev-9.2 libpq-dev

$ sudo pg_createcluster 9.2 main --start

First of all you have to set a new password for the superuser postgres. In order to do so, type the following command line into your terminal

$ sudo -u postgres psql postgres

Now you are in psql, the command interface for postgresql, and in the database postgres. In your terminal it looks like this at the moment

$ postgres=#

To change your password:

$ \password postgres

and type your new password and re type when asked for it.

Create a database (for testing)


If you want to create a db, posgresql has to know, which user you are. Therefore you have to type -u username in front of the command createdb. If you type the following, it means that you as the user postgres want to create a database wich is called mydb.

$ sudo -u postgres createdb mydb

System Kernel tuning


$ sudo nano /etc/sysctl.conf

Add the parameter

kernel.shmmax=8589934592 (8G * 1024 * 1024 * 1024)

/sbin/sysctl -p

Tune Postgres configuration.

$ sudo nano /etc/postgresql/9.2/main/postgresql.conf

Follow my other blog on tuning.

Tuesday, November 24, 2015

Install Latest Webmin in Ubuntu 14.04 From Official Repository

This tutorial shows how to install the latest version of Webmin in Ubuntu Linux from its official repository.
To get started, login your remote server and follow the steps below:
1. Run below command to edit the source file:
sudo nano /etc/apt/sources.list
2. Add this line into the end:
deb http://download.webmin.com/download/repository sarge contrib
Press Ctrl X to exit edit. Press Y to save the changes.
3. Now execute command to download and install the key:
wget -q http://www.webmin.com/jcameron-key.asc -O- | sudo apt-key add -
4. After that, you can always use below commands to install the latest version of Webmin:
sudo apt-get update

sudo apt-get install webmin
Finally in your client’s web browser go to the webmin login page https://ubuntu-serverip:10000

DHIS2 Server setup

DHIS2 Server setup

Here is a useful chapter of DHIS2 official documentation on how to setup a DHIS2 server. This was the documentation of version 13. I like that one because there are some optional setting wich is quite useful. You can find and download the article at dhis2.org.
Server setup
This section describes how to set up a server instance of DHIS 2 on Ubuntu 12.04 64 bit with PostgreSQL as database system and Tomcat as Servlet container. The term invoke refers to executing a given command in a terminal.
For a national server the recommended configuration is a quad-core 2 Ghz processor or higher and 12 Gb RAM or higher. Note that a 64 bit operating system is required for utilizing more than 4 Gb of RAM, the Ubuntu 12.04 64 bit edition is thus recommended.
For this guide we assume that 8 Gb RAM is allocated for PostgreSQL and 8 GB RAM is allocated for Tomcat/JVM, and that a 64-bit operating system is used. If you are running a different configuration please adjust the suggested values accordingly! We recommend that the available memory is split roughly equally between the database and the JVM. Remember to leave some of the physical memory to the operating system for it to perform its tasks, for instance around 2 GB. The steps marked as optional, like the step for performance tuning, can be done at a later stage.
Create new user (optional)
You might want to create a dedicated user for running DHIS - it is not recommended to run as the root user. Create a new user called dhis by invoking 

sudo useradd -d /home/dhis -m dhis -s /bin/bash 

If there is no admin group already you must create it by invoking 

sudo groupadd admin 

Then make the user able to perform operations temporarily as root by invoking 

sudo usermod -G admin dhis 

Then invoke 

sudo passwd dhis 

to set the password for your account. Make sure you set a strong password with at least 15 random characters. You might want to disable remote login for the root account for improved security by invoking 

sudo passwd -l root
Operating system kernel tuning
These settings are optional except for the shared memory setting which is required for PostgreSQL memory allocation. Open the kernel configuration file by invoking 

sudo nano /etc/sysctl.conf 

At the end of the file add the following lines and save.
kernel.shmmax = 4294967296
net.core.rmem_max = 8388608
net.core.wmem_max = 8388608

Make the changes take effect by invoking 

sudo sysctl -p
Install Java
Install Java by invoking the following:
sudo apt-get install openjdk-7-jdk

Check that your installation is okay by invoking 

java -version
Install PostgreSQL
To install PostgreSQL version 9.2 we first need to add a Ubuntu package repository to our system.
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:pitti/postgresql
sudo apt-get update

Now install PostgreSQL by invoking 

sudo apt-get install postgresql-9.2
Switch to the postgres user by invoking 

sudo su postgres
Create a non-privileged user called dhis by invoking 

createuser -SDRP dhis

Enter a secure password at the prompt. Create a database by invoking 

createdb -O dhis dhis2

Return to your session by invoking exit You now have a PostgreSQL user called dhis and a database called dhis2.

PostgreSQL Performance tuning

Do performance tuning by opening the following file by invoking
sudo nano /etc/postgresql/9.2/main/postgresql.conf
and set the following properties:
shared_buffers = 3200MB
Determines how much memory should be allocated exclusively for PostgreSQL caching. This setting controls the size of the kernel shared memory which should be reserved for PostgreSQL. Should be set to around 40% of total memory dedicated for PostgreSQL.
work_mem = 20MB
Determines the amount of memory used for internal sort and hash operations. This setting is per connection, per query so a lot of memory may be consumed if raising this too high. Setting this value correctly is essential for DHIS 2 aggregation performance.
maintenance_work_mem = 256MB
Determines the amount of memory PostgreSQL can use for maintenance operations such as creating indexes, running vacuum, adding foreign keys. Incresing this value might improve performance of index creation during the analytics and data mart generation processes.
effective_cache_size = 8000MB
An estimate of how much memory is available for disk caching by the operating system (not an allocation) and is used by PostgreSQL to determine whether a query plan will fit into memory or not. Setting it to a higher value than what is really available will result in poor performance. This value should be inclusive of the shared_buffers setting. PostgreSQL has two layers of caching: The first layer uses the kernel shared memory and is controlled by the shared_buffers setting. PostgreSQL delegates the second layer to the operating system disk cache and the size of available memory can be given with the effective_cache_size setting.
checkpoint_segments = 32
PostgreSQL writes new transactions to a log file called WAL segments which are 16MB in size. When a number of segments have been written a checkpoint occurs. Setting this number to a larger value will thus improve performance for write-heavy systems such as DHIS 2.
checkpoint_completion_target = 0.8
Determines the percentage of segment completion before a checkpoint occurs. Setting this to a high value will thus spread the writes out and lower the average write overhead.
wal_buffers = 16MB
Sets the memory used for buffering during the WAL write process. Increasing this value might improve throughput in write-heavy systems.
synchronous_commit = off
Specifies whether transaction commits will wait for WAL records to be written to the disk before returning to the client or not. Setting this to off will improve performance considerably. It also implies that there is a slight delay between the transaction is reported successful to the client and it actually being safe, but the database state cannot be corrupted and this is a good alternative for performance-intensive and write-heavy systems like DHIS 2.
wal_writer_delay = 10000ms
Specifies the delay between WAL write operations. Setting this to a high value will improve performance on write-heavy systems since potentially many write operations can be executed within a single flush to disk.
Restart PostgreSQL by invoking 

sudo /etc/init.d/postgresql restart

Set the database configuration

The database connection information is provided to DHIS 2 through a configuration file called hibernate.properties. Create this file and save it in a convenient location. A file corresponding to the above setup has these properties:
hibernate.dialect = org.hibernate.dialect.PostgreSQLDialect
hibernate.connection.driver_class = org.postgresql.Driver
hibernate.connection.url = jdbc:postgresql:dhis2
hibernate.connection.username = dhis
hibernate.connection.password = xxxx
hibernate.hbm2ddl.auto = update

A common mistake is to have a white-space after the last property value - make sure there is no white-space at the end of any line. Also remember that this file contains the clear text password for your dhis2 database so needs to be protected from unauthorized access. To do this invoke 

chmod 0600 hibernate.properties 

which ensures that only the dhis user which owns the file is allowed to read or write to it.
Install Tomcat
Download the Tomcat binary distribution from http://tomcat.apache.org/download-70.cgi A useful tool for downloading files from the web is wget. Extract to a convenient location. This guide assumes that you have navigated to the root directory of the extracted archive. Download the latest stable DHIS WAR file by invoking:
wget stable.dhis2.org
Clear the pre-installed web applications by invoking rm -rf webapps/* Move the DHIS WAR file to the webapps directory and rename it to ROOT.war.
Create a file bin/setenv.sh and add the lines below. The first will set the location of your Java Runtime Environment, the second will dedicate memory to Tomcat and the third will set the location for where DHIS 2 will search for the hibernate.properties configuration file. Please check that the path the Java binaries are correct as they might vary from system to system, e.g. on AMD systems you might see /java-7-openjdk-amd64 Note that you should adjust this to your environment:
export JAVA_HOME='/usr/lib/jvm/java-7-openjdk'
export JAVA_OPTS='-Xmx7500m -Xms4000m -XX:MaxPermSize=500m -XX:PermSize=300m'
export DHIS2_HOME='/home/dhis/config'
If you need to change the port of which Tomcat listens for requests you can open the Tomcat configuration file /conf/server.xml, locate the element which is not commented out and change the port attribute value to the desired port number.
To monitor the behavior of Tomcat the log is the primary source of information. The log can be easily viewed with the command tail -f logs/catalina.out
Set server time
Remember to set the server date correctly according to the time zone of which the main part of the users of the system are located. This will affect e.g. when scheduled tasks are executed. For instance, to the set time to 22:45, October 17, 2012, invoke sudo date 101722452012
Run DHIS 2
Make the startup script executable by invoking chmod 755 bin/* DHIS 2 can now be started by invoking bin/startup.sh The log can be monitored by invoking tail -f logs/catalina.out DHIS 2 can be stopped by invoking bin/shutdown.sh Assuming that the WAR file is called ROOT.war, you can now access your DHIS instance at http://localhost:8080