Monday, 22 June 2015

Steps to Configure HCatalog


HCatalog is the sub module of HIVE. By default, HCatalog will be part of HIVE binaries. In order to work with HCatalog, remote meta store is required to be configured for HIVE meta data definition.

Step:1
First update the environment variables at .bashrc file related to PIG, HIVE and HCatalog as below.

export HIVE_HOME=/usr/lib/hive-0.13.1-bin
export HCAT_HOME=/usr/lib/hive-0.13.1-bin/hcatalog
export PIG_HOME=/usr/lib/pig-0.12.0
export PATH=$PATH:$HCAT_HOME/bin
export PATH=$PATH:$PIG_HOME/bin
export PIG_CLASSPATH=$HCAT_HOME/share/hcatalog/hive-hcatalog-core-0.13.1.jar:\
$HCAT_HOME/share/hcatalog/hive-hcatalog-pig-adapter-0.13.1.jar:\
$HIVE_HOME/lib/hive-metastore-*.jar:$HIVE_HOME/lib/libthrift-*.jar:\
$HIVE_HOME/lib/hive-exec-*.jar:$HIVE_HOME/lib/libfb303-*.jar:\
$HIVE_HOME/lib/jdo-api-*.jar:$HIVE_HOME/conf:$HADOOP_HOME/conf

Step 2

It is required that Hive metastore should be running in remote mode so that Meta Store client knows where is the meta store in $HIVE_HOME/conf/hive-site.xml 

Add or edit the hive.metastore.uris property as follows at hive-site.xml file.

<property>
  <name>hive.metastore.uris</name>
  <value>thrift://<hostname>:9083</value>
</property>

And, run

>> hive --service metastore &

and test if it is running through

>> netstat -an | grep 9083

Step 3

# Create a table using hcatalog

>> hcat -e "create table hcatemptest(Name string, Salary int, Location string) row format delimited fields terminated by ',' stored as textfile"


# Get the schema for a table

>> hcat -e "desc hcatemptest"
OK
Name      string
Salary     int
Location  string