PROVE IT !!If you know it then PROVE IT !! Skill Proficiency Test

Hive Function Cheat Sheet

Intro

HIVE FUNCTION META COMMANDS

  • SHOW FUNCTIONS– lists Hive functions and operators
  • DESCRIBE FUNCTION [function name]– displays short description of the function
  • DESCRIBE FUNCTION EXTENDED [function name]– access extended description of the function

string

TYPES OF HIVE FUNCTIONS

  • UDF– is a function that takes one or more columns from a row as argument and returns a single value or object. Eg: concat(col1, col2)
  • UDTF— takes zero or more inputs and and produces multiple columns or rows of output. Eg: explode()
  • Macros— a function that uses other Hive functions.

HOW TO DEVELOP UDFS

package org.apache.hadoop.hive.contrib.udf.example;

import java.util.Date;
import java.text.SimpleDateFormat;
import org.apache.hadoop.hive.ql.exec.UDF;

@Description(name = "YourUDFName",
    value = "_FUNC_(InputDataType) - using the input datatype X argument, "+
            "returns YYY.",
    extended = "Example:\n"
             + "  > SELECT _FUNC_(InputDataType) FROM tablename;")

public class YourUDFName extends UDF{
..
  public YourUDFName( InputDataType InputValue ){
    ..;
  }

  public String evaluate( InputDataType InputValue ){
    ..;
  }
}

HOW TO DEVELOP UDFS, GENERICUDFS, UDAFS, AND UDTFS

  • public class YourUDFName extends UDF{
  • public class YourGenericUDFName extends GenericUDF {..}
  • public class YourGenericUDAFName extends AbstractGenericUDAFResolver {..}
  • public class YourGenericUDTFName extends GenericUDTF {..}

HOW TO DEPLOY / DROP UDFS

At start of each session:

ADD JAR /full_path_to_jar/YourUDFName.jar;
    CREATE TEMPORARY FUNCTION YourUDFName AS 'org.apache.hadoop.hive.contrib.udf.example.YourUDFName';

At the end of each session:

DROP TEMPORARY FUNCTION IF EXISTS YourUDFName;

Add a Comment

Your email address will not be published. Required fields are marked *