Tuesday, July 19, 2016

Installing Cloudera Manager 5.4.1 in VirtualBox/Linux/CentOS

A step by step guide to install Cloudera Manager in VirtualBox for a clean installation


Step 1 - OS Installation: 

This is a guide to install Cloudera Manager 5.4.1 in Oracle VirtualBox. The first and fore most step is to install OS on the VirtualBox. In this tutorial, we use CentOS 6.6; Cloudera Manager supports the following operating systems (source  cloudera website):

  • RHEL-compatible
    • Red Hat Enterprise Linux and CentOS
      • 5.7, 64-bit
      • 6.4, 64-bit
      • 6.5 in SE Linux mode
      • 6.5, 64-bit
      • 6.6, 64-bit
    • Oracle Enterprise Linux with default kernel and Unbreakable Enterprise Kernel, 64-bit
      • 5.6 (UEK R2)
      • 6.4 (UEK R2)
      • 6.5 (UEK R2, UEK R3)
      • 6.6 (UEK R3)
  • SLES - SUSE Linux Enterprise Server 11, 64-bit. Service Pack 2 or later is required for CDH 5, and Service Pack 1 or later is required for CDH 4. To use the embedded PostgreSQL database that is installed when you follow Installation Path A - Automated Installation by Cloudera Manager, the Updates repository must be active. The SUSE Linux Enterprise Software Development Kit 11 SP1 is required on hosts running the Cloudera Manager Agents.
  • Debian - Wheezy (7.0 and 7.1), Squeeze (6.0) (deprecated), 64-bit
  • Ubuntu - Trusty (14.04), Precise (12.04), Lucid (10.04) (deprecated), 64-bit
Note:
  • Debian Squeeze and Ubuntu Lucid are supported only for CDH 4.
  • Using the same version of the same operating system on all cluster hosts is strongly recommended.

Make sure the OS is installed properly in the VirtualBox before proceeding into the installation of Cloudera Manager.

Step 2 - Disable SELinux:

Selinux value should be changed from ‘enforcing’ to ‘disabled’. If it is ‘enforcing’, the Cloudera Manager will not install and it will ask to disable it when the installation starts. So, the following steps can be used to disable it. 

  • [talend@ClouderaCDH5 ~]$ su –
    • Password: <Your Root Password>
  • [root@ClouderaCDH5 ~]$ cat /etc/sysconfig/selinux – Check whether selinux is disabled or not.
  • [root@ClouderaCDH5 ~]$ vi /etc/sysconfig/selinux – If it is not disabled, open vi editor to edit it.
  • [root@ClouderaCDH5 ~]$ cat /etc/sysconfig/selinux – Check the value for selinux and Restart.


Step 3 - Disable Firewall:

Disabling the Linux firewall is the next step. Cloudera Manager, CDH components, managed services, and third-party components use the ports listed in the tables that follow. These ports need to be opened on each system. If a firewall is being used, such as iptables, and cannot open all the listed ports,  the firewall can be completely disabled to ensure full functionality.


 Disable iptables:
To disable the firewall on each host in your cluster, perform the following steps on each host.
1.    Save the existing iptables rule set.
 iptables-save > /root/firewall.rules
2.    Disable iptables.
o    For RHEL, CentOS, Oracle, and Debian:
chkconfig iptables off
and
/etc/init.d/iptables stop


Step 4 – Edit hosts:
In the next step, we need to edit the hosts file. Copy the hostname from the following step and edit it in the /etc/hosts file.
  • [talend@ClouderaCDH5 ~]$ hostname
    • ClouderaCDH5 – Your Host name will be displayed; Copy this.
  • [talend@ClouderaCDH5 ~]$ vi /etc/hosts

Paste your hostname in the file entry. Refer the below figure for reference.
Step 5 – Download and Install Cloudera Manager
Follow the below steps to get the installer file from the Cloudera repository and install

This will download the installation file to your local directory; Now change the permission to execute the file.
  • [root@ClouderaCDH5 ~]$ chmod u+x cloudera-manager-installer.bin

Once the permission is changed, your installation file will be changed to green color. You can see through ls command like below. Now it is ready to install the cloudera manager in your CentOS.

Run the file to install Cloudera Manager.
  • [root@ClouderaCDH5 ~]$ ./cloudera-manager-installer.bin



So once we run the ./cloudera-manager-installer.bin file, a pop up window will be opening up and click next buttons to proceed till the end the installation. At the end, you can see a pop up window open up and a browser will be open with the url ‘http://localhost:7180’; the user name and password are ‘admin/admin’. If your page is not loading with this URL, it means that cloudera scm server is yet not started. So wait for sometime and look at the log in the next section Step 6 - Look at the Log.

You can see the below screen on the successful installation.

Step 6 – Look at the Log file
Once the installation completes, we can check the log file for the server starts. On a successful starting of the Cloudera-SCM-Server, you can see a message ‘Finished constructing repo:’ in your log file. Now the URL localhost:7180 will be opening.
  •  [root@ClouderaCDH5 ~] $ tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log


Step 7 – Cluster Installation
It is time to install the Cluster. Once your login to the http://localhost:7180; Cluster installation wizard will be coming up. Accept the license and click on Continue.


Now choose the edition of Cloudera you want to deploy. In this installation, I have chosen ‘Cloudera Express’.


Specify the hosts for your Cluster; just search with the hostname and select.

Now it’s time to choose the repository. If you want to install the most recent version of Cloudera Manager, then select Use Parcles (Recommended) option. If you want to install a specific release, then choose Use Packages and select the specific release.


If you want to enable single user mode, select this option; otherwise continue.

Now the Cluster Installation begins as below. All required packages and items will be installed in this step.

 Once the installation completes, you can see the below screen with green color. Click Continue.

Step 8 – Cluster Setup
There will be ­­­multiple options displayed in Cluster Setup. Choose the one you want to have in your environment and Continue.

In the next step, database will be configured for the services you selected earlier. In this installation, database setup on Hive and Oozie Server is required for the service Core with Spark. Cloudera Manager will provide a password by default. Just note it down in a text file; this may be useful to debug if you encounter any connection issues.

When you click on the button Test Connection, you will receive a message ‘Skipped’; this will be cleared after the clean installation. So proceed with Continue.

Now this is the last step in Cluster Setup. This step is called First Run; In this step, the following services and packages will be configured and started in the follwo order.
1.    Deploying Client Configuration like hdfs-site.xml, core-site.xml etc
2.    Starting HDFS
3.    Starting MapReduce and Yarn
4.    Starting Spark
5.    Starting Hive
6.    Starting Oozie Server
7.    Starting Hue

The First Run Command will start like this. Status is Running.


And this will end like this. Once all the configured services and packages started, you will see the screen like this. You can see the Status Finished. 


This is end of instillation. Click Finish to see the cluster details.


These are all the steps to install Cloudera Manager properly. If you guys face any difficulties in installation, feel free to write to us from the below link; we will help you. You can follow us on the below to get to know more about us.

 We are data integration experts at Pacific Big Data, providing solutions to fit your needs.


 Contact us at:








Posted by at No comments:
Subscribe to: Comments (Atom)