This procedure is for QuerySurge Agents deployed on Windows. The drivers on which this procedure is based are the Cloudera Hive JDBC drivers, which may be downloaded from the Cloudera website. We strongly encourage using an updated version of these drivers. The setup here assumes that your Hive server uses Kerberos authentication with an LDAP server in the background.
1. JDBC Driver files and related files
1a. Download the Cloudera driver zip, and select the JDBC 4.1 drivers. The "Cloudera JDBC Driver for Apache Hive" pdf that comes with the driver download has a full description of the driver setup and options; this article is based in part on these instructions. The driver consists of multiple jar files; you will need to distribute all the jar files to each QuerySurge Agent.
Deploy the driver jars to your Agent; you can find instructions for this procedure here.
Note: If you have previously deployed Hive driver jars, you should delete them from the Agent driver directory before you deploy the driver jars from your Hive server.
2. Kerberos Configuration Files
2a. krb5.conf (krb5.ini) file - From your Kerberos admin or other knowledgeable resource, obtain a krb5.conf file. On Windows, the file may be called krb5.ini.
3. Agent Setup
Note: We recommend using the QuerySurge Agent directories for the file locations in this setup step, since this should be the same for all your Agents. However, there is no requirement to use these specific directories.
3a. Go to the directory: <QuerySurge Install Dir>\QuerySurge\agent\, and run QuerySurgeAgentw.exe as Administrator. (Make sure to run QuerySurgeAgentw.exe and not QuerySurgeAgent.exe.) On the General tab, use the Stop button (lower left) to stop the Agent.
3b. In <QuerySurge Install Dir>\QuerySurge\agent\jdbc, delete any hadoop or hive jar files.
Copy the jar files you downloaded (step 1a) to this directory.
3c. Copy your krb5.conf (or krb5.ini) file to <QuerySurge Install Dir>\QuerySurge\agent\.
3d. Create a Windows environment variable for the Kerberos cache. Right-click on the computer icon and select Properties. Click on Advanced System Settings. In the System Properties dialog, click on the Advanced tab. Click on the Environment Variables button. Under System Variables, click New. In the New System Variable dialo, type KRB5CCNAME in the Variable Name field. In the Variable Value field, type the path for your credential cache file (e.g. C:\TEMP\krb5cache). Click the OK buttons to save your changes. A system restart is advisable.
3e. You may need to run a
kinit command locally, on your Agent box(es). You can use the kinit command that is part of the Java distribution that comes with QuerySurge (found at: <QuerySurge Install Dir>\QuerySurge\java\bin\kinit.exe). Or, you may install a KDC client on the Agent box(es). Not all clients are compatible with all Kerberos implementations, so this should be done in consultation with a resource knowledgeable about your organizations's Kerberos setup.
4. Agent Configuration
4a. In QuerySurgeAgentw.exe, click on the Java tab.
In the Java Options box, add the following on separate lines. Do not delete anything from the Java Options box! Note that some of the values are file paths, which are dummy paths in the sample below:
-Djava.security.krb5.conf=<QuerySurge Install Dir>\QuerySurge\agent\krb5.ini
Note: You'll have to replace the dummy paths in these settings with actual paths.
Note: During setup, a high level of debug output is often helpful. Add the following command-line options while you are debugging your connection:
4b. In QuerySurgeAgentw.exe, click on the General tab.
Use the Start button (lower left) to start your Agent.
5. QuerySurge Connection Wizard (using the Connection Extensibility option)
5a. Open the Connection Wizard in the QuerySurge Admin view. Select the Connection Extensibility option in the Data Source dropdown. Use the following values and templates:
Hive JDBC URL:
A simple sample dummy Hive URL:
Once you have set up a URL, try a Connection Test or a QueryPair with a simple query of the form:
SELECT * FROM mydatabase.mytable LIMIT 5
Notes and General Comments:
- You may need to deploy unlimited strength policy jars to your QuerySurge java. (An error output of: found unsupported keytype (18) may indicate the need for unlimited strength jars.) For Java 7, these jars (local_policy.jar, US_export_policy.jar) are available here. The jars should be deployed to <QuerySurge Install Dir>\QuerySurge\java\lib\security. The existing policy jars should be cached in another folder.
- The Connection test is subject to the browser timeout. If you get a timeout message during the Connection test, that may not be an indication of a true timeout. If you receive a timeout message, create a QueryPair using your test query for both Source and Target queries.
- Kerberos has many possible ways that it can be set up. These instructions go through a common path, but it may not be completely correct for your environment. You will likely have to consult with a Kerberos admin or other knowledgeable resource on this setup.
- This procedure should be performed on all Agents that you have deployed.