Articles in this section

See more

Agents

Follow

Versions:1.0 - 7.2

QuerySurge Agents perform the actual query tasks by QuerySurge. Agents execute queries against Source and Target data stores, and return the results to QuerySurge.

Agent Installation

  1. Install the QuerySurge Agent using the QuerySurge Installer.
    Note: the Agent version must match the QuerySurge App Server version. Therefore, you must use the same installer/version to install Agents as you used to install the QuerySurge App Server.

  2. You may install multiple QuerySurge Agents up to the number that your license permits.

  3. Each QuerySurge Agent deployed in your environment needs at least one JDBC driver to work with your data sources. A JDBC driver is a piece of software that allows your Agent to "talk" to a data source. So, for example, if you plan to use QuerySurge with Oracle and Teradata, you will need to deploy both the Oracle and the Teradata drivers to each Agent.

  4. The QuerySurge Installer comes bundled with JDBC drivers for several industry-standard databases, and you can install these drivers when you install the Agent.

  5. You can always add new drivers using the installer after you have installed an Agent.

  6. Your JDBC driver may not be bundled with QuerySurge; if not, you can deploy the JDBC driver(s) manually to the Agent machine. For more information, see Adding JDBC Drivers.

  7. Please note that the QuerySurge Agent installs as a service.

  Note: Agents should be installed on wire-connected platforms on the same network as your QuerySurge App server. Installing Agents on machines that might be connected via WiFi or by VPN will not only send potentially large amounts of data over a narrow connection but may also expose data on an insecure connection. Additionally, network topology, especially over VPN, may slow or interfere with the connection.

Note: Agents should never be co-installed with other programs or processes that might interfere with or hook the Agent process. This includes, but is not limited to, co-installation with products like automated GUI-testing tools.

For a more detailed articles on Agent installation please see either of the following:

The QuerySurge Agent as a Service

The QuerySurge Agent is installed as a service. This means that:

  • The Agent will start automatically when its' deployment platform is rebooted.
  • The Agent runs regardless of whether any user is logged in or not.

The Agent Service Console (for Agents on Windows)

  • The Agent Service Console gives you access to key Agent configuration settings and to the Agent Status. Launch the Console using the Start menu: All Programs > QuerySurge > Open Agent Console.

 

  • You can adjust Agent configuration values using the Agent Service Console. Remember to re-start the Agent after making changes. See Configuring a QuerySurge Agent for details.
  • The Service Console also shows the Agent Status ("Started" or "Stopped").

    Note: The Agent Status is a simple, high-level indicator of whether the Service is running or not, and does not reflect whether the Agent has an error condition. To see detailed information about an Agent's condition, choose File > Show Current Log to see the current Agent log.

  • The Agent Service will have the default permissions of the user under whose login the Agent was installed. Usually, this login is that of a local administrator, so the Agent is usually installed with local administrator rights.
  • Note that a local administrator may not have permissions to domain resources, such as network share drives, which can affect Agent access to such resources. If you plan to use your Agent to query flat files on shared drives, you may have to set some additional permissions for the Agent Service. See Changing Agent Service Permissions for Network Resources.

Adding JDBC Drivers

Users can "manually" add JDBC drivers to Agents at any time. Details for this process can be found in the following Knowledge Base articles:

QuerySurge Agents on Microsoft Windows

QuerySurge Agents on Linux

Registering an Agent with QuerySurge

QuerySurge Agent Auto-Registration

The QuerySurge Agent will "auto-register" itself with QuerySurge. When you install a QuerySurge Agent, the installer will ask you for the URL of your QuerySurge installation. The installer will then install and start the Agent. When the Agent first starts, it will check its registration status with QuerySurge, and it will auto-register if necessary. QuerySurge must be installed an running for this process, of course.

Note: The QuerySurge Installer will only ask you for the QuerySurge URL if you are installing the Agent on a different box than you are installing the QuerySurge App Server on.

As a security measure, once your Agent has auto-registered, a QuerySurge Administrator must log into QuerySurge and Enable the newly registered Agent, by clicking on the checkbox in the Enabled column for that Agent.

Registering an Agent Manually

In the Administrative View, choose the Agent node in the tree. Click on the Add button to launch the Add Agent dialog. To obtain the Agent Hostname, launch the Agent Service Console; right-click on the Agent Name and select Copy from the popup menu. You can paste this into the Hostname field in the Add Agent dialog. No connection between the Agent and the QuerySurge server is required for this copy/paste setup step.

Basic Registration

Basic configuration settings are set by the QuerySurge Administrator when the Agent is registered with QuerySurge:

  • Thread Pool Size - the total number of threads the Agent can use to execute queries. The optimal number will depend on the resources available to your Agent platform. Three (3) threads is a reasonable number to start experimenting with.

    Note: As rough benchmark figures, we regularly use 3 threads on 2 GB RAM Agent deployments, and 5 threads on boxes with 4 GB or greater.

    Note: There is a tradeoff between launching a larger number of query threads vs. the cpu and memory resources required to service those threads while they are executing. At a certain point, more threads can start to slow the Agent down, and increasing the number of threads will eventually lead to lower throughput, not higher. Because Agent performance has Operating System and hardware dependencies, a bit of experimentation in your environment is worthwhile.

  • Maximum Thread Polling Interval - The maximum interval the QuerySurge Agent will wait to poll QuerySurge for query instructions in milliseconds.

    Note: Typical values for this setting are 300,000 ms (5 min).

  • Test Suite Retrieval Behavior - This governs QuerySurge Agent behavior regarding retrieval of Suites. The first option lets the Agent pick up any Suites whether specifically assigned to that Agent or not; the second option lets the Agent retrieve only Suites specifically assigned to that Agent.

 

Configuring a QuerySurge Agent

Configuration Options

QuerySurge Agents have multiple configuration options that can affect how your Agents behave. You can configure your Agents via the QuerySurge Agent Service Console. On Microsoft Windows, launch the Agent Service Console from the Start menu: All Programs > QuerySurge > Open Agent Console.

Basic Configuration

In order to connect your Agent(s) to QuerySurge, the Agent requires the QuerySurge URL. In many cases, the Agent has been configured with the URL during installation; if it has not, you will have to configure it manually. The URL format is:
          http://<server>:<port>
Once you have entered the URL, press the 'Start Agent Service' button to connect the Agent to QuerySurge. If the Agent service is already started, press the 'Stop Agent Service' button, edit the URL, and then press the 'Start Agent Service' button.

Advanced Configuration

The following configuration options are available via the Agent Service Console

  • Listen for Special Tasks
  • Agent Message Size
  • Agent Fetch Size
  • Agent Heap Size

Listen for Special Tasks

This setting controls whether the Agent listens for tasks such as Connection Test tasks. The purpose of this setting is to let you 'turn off' certain tasks once they are completed - for example, once your Connections are set up, you won't need all your Agents checking whether any Connection Tests need to be performed. Turing these task requests off cuts down on unnecessary network traffic.

Agent Message Size

This setting gives you control over the maximum message size (in MB) that is sent from the Agent to QuerySurge. The Agent Message Size can have values of 'No Limit' or numeric values. By default, the Agent Message Size is set to 3 MB. Selecting the 'No Limit' setting tells the Agent to use the maximum size allowed, which is a bit under 50 MB. Depending on the memory resources available on your Agent platform, you may want to experiment with message sizes smaller than the maximum. Using smaller message sizes will cause the Agent to send its data in a larger number of smaller messages, saving Agent memory at the expense of a larger number of messages (= greater network traffic).

Note: This setting plays a significant role in Agent heap memory consumption. This, in turn, affects the maximum size Resultset that your Agent can handle with the memory that is available to it.

Agent Fetch Size

This setting allows you to set the JDBC fetch size, in ResultSet rows. According to the JDBC documentation, the fetch size is "the number of rows that should be fetched from the database when more rows are needed for this ResultSet." Selecting 'No Limit' lets the driver make its own choice about how many rows to fetch. Note that according to the JDBC specification, this is only a hint, and driver vendors are permitted to implement this feature at their own discretion, and need not implement it at all. Consult your JDBC driver documentation to see the specifics of how your drivers implement the JDBC fetch size.

Agent Heap Size

This sets the maximum amount of memory that can be allocated to the Java heap. This is a critical parameter, since it governs the heap that the Agent has access to during query execution and processing.

Note: A significant factor in Agent heap memory consumption is how your JDBC driver manages memory. This, in turn, affects the maximum size Resultset that your Agent can handle with the memory that is available to it. There can be significant variation in JDBC drivers here, so it is wise to perfom trials with your driver to see how the Agents behave on the machines you have deployed them on.

There are a couple of important caveats for setting this value:

  • Heap size can only be set at launch, so if you choose a new value, you will receive notification that the Agent must be stopped and re-launched.
  • The heap size setting refers to contiguous memory. If your system does not have the amount of memory requested as contiguous memory, the Agent process will immediately exit.

    Note: If, on Agent launch, the Agent quits and the log file contains the following error: "Error occurred during initialization of VM. Could not reserve enough space for object heap", then the Operating System could not give the JVM the heap size it requested. To set the heap size request lower so that your Agent will launch, you must edit (use any text editor) the Agent config file (/../config/agentconfig.xml); set the <heap> tag value to a lower value than it is currently set at (1000 MB is the default; this works on most systems), save the file, and re-launch the Agent.

    Note: If your system is only able to support heap sizes in the hundreds of MB, then the Resultset sizes that your Agent can handle are likely to be relatively small.

    Note: The Agent Message Size, Agent Fetch Size and Agent Heap Size all interact to shape the size of the queries that your Agent can manage. The QuerySurge Agent installs with default values that are optimized for a generic set of queries; these optimizations are a starting point for work with your queries. Tuning these values using a set of your standard queries is suggested.

The QuerySurge Agent and HTTPS/SSL

See this Knowledge Base article for details about HTTPS/SSL setups for QuerySurge components.

Troubleshooting

Troubleshooting issues are discussed in this Knowledge Base article: Troubleshooting QuerySurge Agent Issues

 

Related articles

Comments

0 comments

Article is closed for comments.