No databases are available. Permissions could be missing error in Hive with Sentry

Apache Sentry is a Big data tool used to enforce fine grained role based authorization to data and metadata on your hadoop clusters. Recently I was playing around with Sentry and from the configuration manual on cloudera website, the integration of Sentry on the hadoop cluster looked like a breeze but there were certain pain-points that sucked away my time for nothing.

Following were the major problems I came across:

1. Error 1: Could not find the option “Create new roles” or “Create new permissions” on Hue UI for Sentry App.

3. Problem 2: Cannot see Hive databases and tables to assign roles and proviles for.

2. Error 3:

No databases are available. Permissions could be missing error in Hive with Sentry

: ** This was the Major Issue for me.

NOTE: There are two ways to enable Sentry on your Hadoop cluster. You could opt to use Sentry Service which is relatively new. Other way is to configure sentry using Policy file wherein you define all your authorization rules in the policy file that is checked every-time.

Add new roles and permissions

Add new roles and permissions

How to configure Sentry Service to work with HDFS/Hive on Hadoop cluster

1. Follow the steps under Before enabling the sentry service

2. If you can’t find the option “Create new roles” or “Create new permissions” on Hue UI for Sentry App, That is because the user which is logged into the hue account does not belong to Sentry admin groups.

As mentioned on the link given,

Add the Hive, Impala and Hue groups to Sentry’s admin groups. If an end user is in one of these admin groups, that user has administrative privileges on the Sentry Server.

Now, Does the user that is logged in HUE belongs to any of the groups you have listed as admin groups in HUE? I am quite sure NO. When I started, I was using the default user that comes configured with HUE. The username was ‘hue’ and it belonged to group named default.

You can configure users and groups in hue easily as an admin user through ‘Manage Users’ tab at the top right . If you can’t find it, follow this link configure users and group with the limited permissions in hue.

But, should I add default group as admin in the sentry app? I could. No issue there. But, the user and group that I am going to be using as admin or a general user, should exist as a real user on your UNIX/LINUX system. Default group does not exist by default on your System. So, you will need to add the group ‘default’ and create a real user under default group with the same username as you have in hue.

Let’s see how to do that on Linux system:

To create a group on Linux , use the command :

groupadd group_Name

To add users to a group

useradd -G group_Name user_Name

Infact, you can also use a tool to sync your users and groups on Linux system with those in HUE so that you don’t have to create them in HUE and on your System individually. There’s this really easy way to synchronize users and groups.

Now, once you have added the same users and groups on your Linux system and in HUE, go ahead and add that group as an Admin group in Sentry app.

<property>  <name>sentry.service.admin.group</name>  <value>hive,impala,hue</value> </property>

After adding the group, the configuration screen looks like this :

Sentry Admin group

Sentry Admin group

[NOTE]: Now, let’s make this user a ‘superuser’ in HUE through the same Users and groups screen in Hue you have accessed before to add new users and groups.

Once you have made your user a superuser in HIVE and have given all possible permissions to this user through hue, Log into hue using this user.

Now, you should be able to add roles and permissions in HUE.Your screen for Sentry App in HUE should look like this now:

Add new roles and permissions

Add new roles and permissions

Cool ? Problem 1 Solved.

Now, You can see the button to assign roles and permissions in HUE, but do you see databases and tables under server1? I guess not.

No databases or tables

No databases or tables

That’s because may be you already enabled Sentry service for Hive/HDFS before you authorized a user in Sentry as super user to be able to access all of the databases and tables, In short everything. Unless a user is authorized to access all databases, tables on the Hive server or all folders and Files on HDFS, How would and who would assign other users the roles and privileges to access the data?

Our problem now is that the user cannot see databases or tables so as to be able to assign permissions to the.

Go to hive configuration and make sure that you have not already enable Sentry for Hive. This is the stupid mistake I actually did and struggled for hours to find out why it was not working!

Solution to Problem 2:

1. Turn off Sentry for Hive.

Disable Sentry for Hive First

Disable Sentry for Hive First

2. Assign a user access to the entire server i.e all tables, all groups and the entire server before you enable the Sentry service. This would be a super user for Sentry that has the privilege to access the server and assign other users roles and privileges.

To do that, Create a role in Hive UI with access to the server (do not go down to the level of databases or tables) and assign it to a group.

This should solve your problem 2 and you should be able to see databases and tables to assign privileges for in Hive.

Huhhhhhhh!

Now, you can assign other users whatsoever roles and privileges you want them to have.

Go back to Hive Configuration and Enable Sentry.

Now, open the Hive editor and try to execute something as simple as show databases;

Does it give you an error that says :

No databases are available. Permissions could be missing error in Hive with Sentry

It looks like this in Hive editor:

no databases are available. Permissions could be missing

no databases are available. Permissions could be missing

This took me the longest to fix. You get this error because you don’t have a strong authentication mechanism enabled like Kerberos or LDAP. It is not enabled by default.

Nowhere does it say that Sentry is not going to work without Kerberos or LDAP enabled. Unfair!

Anyway, Go ahead and enable Kerberos or LDAP for an authentication mechanism on your hadoop cluster. How to enable Kerberos or LDAP is beyond the scope of this article but here are a few links :

For external authentication on cloudera : Refer https://www.cloudera.com/documentation/enterprise/5-5-x/topics/cm_sg_external_auth.html .

However, if you do not want to enable LDAP or kerberos. Add the following property to the HiveServer2 and Hive metastore’s sentry-site.xml:

<property>  <name>sentry.hive.testing.mode</name>  <value>true</value></property>

and it will work after this. That puts an end to errors. Cool!

I hope this helped you save some time.

P.S: Internet is a successfully thriving community from generous contributions of people from across the globe. You can help it thrive too. Please contribute. As a reader, you can be contributing with your valuable feedback.
Please drop by a comment or Share to let people know you were here. ๐Ÿ™‚ ๐Ÿ™‚

16 thoughts on “No databases are available. Permissions could be missing error in Hive with Sentry

  1. Hi,

    fantastic guide!

    I have a problem for solution 2, when i try to add new role i received this error:

    Unknown error for request: TCreateSentryRoleRequest(protocol_version:2,
    requestorUserName:hive, roleName:test),
    message: Error(s) were found while auto-creating/validating the datastore for classes.
    The errors are printed in the log, and are attached to this exception.

    Can you help me?

  2. thanks for your quickly feedback, this is the log:

    [03/Nov/2016 09:53:45 +0100] hive ERROR could not create role
    Traceback (most recent call last):
    File “/opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hue/apps/security/src/security/api/hive.py”, line 176, in create_role
    api.create_sentry_role(role[‘name’])
    File “/opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hue/desktop/libs/libsentry/src/libsentry/api.py”, line 49, in decorator
    raise e
    SentryException: Unknown error for request: TCreateSentryRoleRequest(protocol_version:2, requestorUserName:hive, roleName:prova), message: Error(s) were found while auto-creating/validating the datastore for classes. The errors are printed in the log, and are attached to this exception.

  3. Hi Simrankaur,

    Firstly thanks for your post! It is clear and easy to follow.

    I’ve followed every step on this post as well as the Cloudera and Hue documentation related to Sentry but cannot seem to get past the second problem you describe. I do not have LDAP or Kerberos configured on my cluster, and have set the Hive service advanced configuration snippet for sentry-site.xml property sentry.hive.testing.mode to true. I can only see server1 “default” db and none of the other Hive databses or tables.

    Could you point me in the right direction?

  4. Hi Nicolle,

    Are you sure you have the group to which your logged in user belongs to sentry.service.admin.group hive,impala,hue,default . But before anything else, you need to disabled Sentry service. After that:

    Assign a user access to the entire server i.e all tables, all groups and the entire server before you enable the Sentry service. This would be a super user for Sentry that has the privilege to access the server and assign other users roles and privileges.

    To do that, Create a role in Hive UI with access to the server (do not go down to the level of databases or tables) and assign it to a group.

    This should solve your problem 2 and you should be able to see databases and tables to assign privileges for in Hive.

    Once you are past this step, you should be able to see the tables and databases.

    Are you sure you’ve performed these steps?

  5. Thanks for your response,

    Yes I performed all of these steps multiple times. It’s very difficult to debug the source of the problem. The role with “ALL” privileges has been granted to the sentry admin user across all of server1, and the superuser and supergroup all exist on the OS as well. I have ensured to specify in the sentry-site that this is an unsecure cluster.

  6. @KK020486 : You’re welcome. Using Hue editor, Go to Administration (on top right, third link from left) and then Manage Users. You can create/delete and update users there.

  7. hi simran,
    Iam unable to set my hive into testing mode. can you please let me know where to set up the property sentry.hive.testing.mode to true? can we do it from CDH manager UI or do we need to change it in xml files? if yes, then is it multiple files we need to change?

  8. Nice post – I struggled for days following Cloudera instructions and videos
    but your instructions got me the furthest along. I can see create and manage Roles, assign them to groups and the superuser can manage them.
    But my assigned users are getting FAILED: SemanticException No valid privileges errors
    I do not have LDAP (or kerberos) and I only the users in HUE to not see certain databases – select from specific db is good enough.
    I don’t want to enable LDAP or replicate the users in HUE on as Linux users either
    I must be missing something

Leave a Reply

Your email address will not be published. Required fields are marked *