PROVE IT !!If you know it then PROVE IT !! Skill Proficiency Test

BDR between kerberos enabled environment Enabling Replication Between Clusters with Kerberos Authentication

Home
> bdr, cloudera > BDR between kerberos enabled environment Enabling Replication Between Clusters with Kerberos Authentication

BDR between kerberos enabled environment Enabling Replication Between Clusters with Kerberos Authentication

To enable replication between clusters, additional setup steps are required to ensure that the source and destination clusters can communicate.

Note: If either the source or destination cluster is running Cloudera Manager 4.6 or higher, then both clusters (source and destination) must be running 4.6 or higher. For example, cross-realm authentication does not work if one cluster is running Cloudera Manager 4.5.x and one is running Cloudera Manager 4.6 or higher.

Continue reading:

  • Considerations for Realm Names
  • Configuration
  • Configuring a Peer Relationship

Considerations for Realm Names 

If the source and destination clusters each use Kerberos for authentication, use one of the following configurations to prevent conflicts when running replication jobs:

  • If the clusters do not use the same KDC (Kerberos Key Distribution Center), Cloudera recommends that you use different realm names for each cluster.
  • You can use the same realm name if the clusters use the same KDC or different KDCs that are part of a unified realm, for example where one KDC is the master and the other is a slave KDC.
  • Note:If you have multiple clusters that are used to segregate production and non-production environments, this configuration could result in principals that have equal permissions in both environments. Make sure that permissions are set appropriately for each type of environment.

Important: If the source and destination clusters are in the same realm but do not use the same KDC or the KDCs are not part of a unified realm, the replication job will fail.

Configuration 

  1. On the hosts in the source & destinationclusters, ensure that the conf file (typically located at /etc/kbr5.conf) on each host has the following information:
    • The kdc information for the sourcecluster’s Kerberos realm. For example:

realms

 INTBDA.BIL.COM = {
kdc =<–KDC__MASTER__NODE–>:88
kdc = <–KDC__SLAVE__NODE–>:88
admin_server = b<–KDC__MASTER__NODE–>:749
default_domain = bnet.luxds.net
}
DEVBDA.BIL.COM = {
kdc = <–KDC__MASTER__NODE–>:88
kdc =<–KDC__SLAVE__NODE–>:88
admin_server = <–KDC__MASTER__NODE–>:749
default_domain = bnet.luxds.net
}

  • Domain/host-to-realm mapping for the sourcecluster NameNode hosts. You configure these mappings in the [domain_realm] For example, to map two realms named SRC.MYCO.COM and DEST.MYCO.COM, to the domains of hosts named hostname.src.myco.com and hostname.dest.myco.com, make the following mappings in the krb5.conf file:

[domain_realm]
.src.myco.com = SRC.MYCO.COM
src.myco.com = SRC.MYCO.COM
.dest.myco.com = DEST.MYCO.COM
dest.myco.com = DEST.MYCO.COM

BE CAREFUL!!!

But in MYcase, the scenario is completely different. Since the domain names are all ending with bnet.luxds.net, it is not possible to use directly given way. If we use it, then we will face below:

PriviledgedActionException as:hdfs/<–HOST_NAME–>>@DEVBDA.BIL.COM (auth:KERBEROS) cause:java.io.IOException: java.lang.IllegalArgumentException:

Server has invalid Kerberos principal:

hdfs/<–HOST_NAME–>>@INTBDA.BIL.COM, expecting: hdfs/<–HOST_NAME–>>@DEVBDA.BIL.COM

To handle this problem, we have to change the domain_realm as below:

From:

[domain_realm]
.bnet.luxds.net = INTBDA.BIL.COM
bnet.luxds.net = INTBDA.BIL.COM

To(when also adding the other cluster information as well). It shoud contain all host that we have for given environment.

[domain_realm]
NODE_1_CLUSTER_1= INTBDA.BIL.COM
NODE_2_CLUSTER_1= INTBDA.BIL.COM
NODE_3_CLUSTER_1= INTBDA.BIL.COM
NODE_4_CLUSTER_1= INTBDA.BIL.COM
NODE_5_CLUSTER_1= INTBDA.BIL.COM
NODE_6_CLUSTER_1= INTBDA.BIL.COM
NODE_1_CLUSTER_2= DEVBDA.BIL.COM
NODE_2_CLUSTER_2= DEVBDA.BIL.COM
NODE_3_CLUSTER_2= DEVBDA.BIL.COM

ihave to arrange domain_realm as above on both cluster.

Trust Creation

addprinc krbtgt/INTBDA.BIL.COM@DEVBDA.BIL.COMto DEV
addprinc krbtgt/DEVBDA.BIL.COM@INTBDA.BIL.COMto INT

With these two,  i will be able to reach clusters.

Add dfs.namenode.kerberos.principal.pattern parameter to all clusters

  1. On the destinationcluster, use Cloudera Manager to add the realm of the source cluster to the Trusted Kerberos Realms configuration property:
    1. Go to the HDFS service.
    2. Click the Configuration
    3. In the search field type “Trusted Kerberos” to find the Trusted Kerberos Realms
    4. Enter the source cluster realm.
    5. Click Save Changesto commit the changes.

Trusted Realm Addition on HDFS

   

Configuring a Peer Relationship
  • Youdao
  • Xian Guo
  • Zhua Xia
  • My Yahoo!
  • newsgator
  • Bloglines
  • iNezha
  • Categories

    TROUG

    bilgic.sercan01@gmail.com

    Join 797 other followers

    Advertisements

    Let’s block ads! (Why?)