Welcome to QuerySurge™
Welcome to QuerySurge. QuerySurge is a testing tool from RTTS that automates the testing of Big Data, Data Warehouses and Migrations as well as Database Upgrades, System Upgrades, Interface testing and the ETL process. Unlike other testing tools, QuerySurge analyzes and pinpoints differences in data to ensure that data extracted from it's sources remains intact in the target, creating a more efficient testing process.
This Installation Guide is meant to serve as a guide through QuerySurge and some of it's key features. Information about other QuerySurge features can be found on our website under Help.
Resources
QuerySurge Help is built into the product and provides immediate answers to questions that arise as you work. Help can be found in both the top and bottom toolbar under the Help icon.
QuerySurge Chat can be accessed through QuerySurge or on the QuerySurge web site. Experts are available Monday through Friday from 9:00am to 5:00pm Eastern Time to assist you with questions regarding installation, configuration and all other QuerySurge related questions. If questions arise during Chat off hours users can utilize text and comment.
Video Library provides tutorials, demos and webinars on the ins and outs of QuerySurge usage: http://www.querysurge.com/product-tour/video-library
Contact Us page provides a form where you can submit comments, questions or suggestions: http://www.querysurge.com/company/contact-us
About the QuerySurge Architecture
QuerySurge is a locally installed, browser-based testing tool. QuerySurge supports all Supporting all current browsers making it perfect for all teams no matter where their located. QuerySurge extracts the data-under-test to its own infrastructure which eliminates processing overhead on the Source and Target database servers in your data warehouse architecture. The QuerySurge architecture is comprised of an application server, a database server and agents.
QuerySurge Application Server and Database
QuerySurge utilizes a Tomcat Application Server and a MySQL database. The database is bundled in the product and embedded within it.
QuerySurge Agents
QuerySurge Agents are the components of the architecture that execute queries against source and target data sources. They then return the results to the QuerySurge database. The Agents execute SQL queries, validating each piece of data throughout the ETL process. This exposes all data mismatch failures, row count differences and column type mismatch failures. This creates a process that tests 100% of data quickly.
QuerySurge Agents do not reside on the physical source or target database boxes even though they issue queries to both the source and target databases. Agents can be deployed on the same box as the QuerySurge Application Server and QuerySurge Database Server in a single box install or on any "satellite" boxes in the environment. They are often test lab boxes or available desktop boxes. When the QuerySurge Agent can run multiple queries simultaneously when it receives a bundle of queries it.
One of the features of QuerySurge is the ability to raise your ‘testing throughput’. This fires off QueryPairs in bulk against your Sources and Targets so that you can execute at the highest level your environment can support. This is possible because the QuerySurge Agent can deploy multiple Agents in your environment on ‘satellite’ boxes surrounding your QuerySurge server. Each Agent can run multiple Query threads. With QuerySurge you can install up to 10 Agents. This means that more agents there are the more queries and more throughput.
You can conclude how many Agents are appropriate for your environment through experimentation. Start with 2 or 3 Agents once you have built an initial Test Library and see how your Sources and Targets behave. Add additional Agents, again monitoring the Source and Target behavior. As you continue to add Agents, the loads on Sources and Targets will grow with query volume – and Source/Target response times will start to grow as well. Identify the level where start times have started to increase and adjust to that level. This level is roughly the maximum throughput your environment can support.
Visit AdministrationAgents in the Help guide for more information on Agents.
Database / Data Warehouse Support
In principle, QuerySurge can support any JDBC-compliant data source. QuerySurge currently ships with Oracle, MySQL, Teradata, Aster, SQL Server, DB2, Informix, Netezza, Sybase, Hadoop, in-memory databases, and flat file support and supports many other data sources.
QuerySurge Installation Best Practices
For trial installations, it is common to install all of the QuerySurge™ components on a single machine. In order to improve performance, most production installations will either:
- Deploy each component on a separate machine
- Deploy all components on a single machine with substantial resources
For the single machine installation procedure, see QuerySurge - Installation Guide for Windows
QuerySurge Installation Requirements
This Installation guide will walk you through QuerySurge™'s installation process. After your installation is complete, you can refer to the connection guides or the Setup Wizard to complete the QuerySurge™ configuration.
What you’ll need:
- Administrative rights on the machine(s) where you’ll be installing on
- The QuerySurge™ installation file for the appropriate architecture (64 bit or 32 bit)
- Connection details for any database or file that you would like to use with QuerySurge™
- Make sure all antivirus software (or similar software) is shut down before you start the installation, as these processes can interfere with the installation process, potentially breaking your installation.
Sample Connection Details:
| Oracle Example | SQL Server Example | ||
| Server Name | systemA.example.com | Server Name | systemB.example.com |
| Port | 1521 | Port | 1433 |
| Service Name | Sales | Database | Schema_A |
| User Name | myuser | User Name | myuser |
| Password | mypass | Password | mypass |
| Netezza Example | Teradata Example | ||
| Server Name | systemC.example.com | Server Name | systemD.example.com |
| Port | 5480 | Port | N/A |
| Database | Schema_C | Database | Schema_C |
| User Name | myuser | User Name | myuser |
| Password | mypass | Password | mypass |
For more information on creating connections, see QuerySurge™ configuration.
Hardware Requirements
For more information on hardware configuration for your deployment, see QuerySurge™ System Requirements.
Support
Have Additional Questions?
The orange Questions? icon can be found on the left hand side of all our web pages, and in the QuerySurge application.
Support is available Monday through Friday, 9am-6pm Eastern Time. If you require help in between active help times you can create a ticket and our team will answer it at soon as possible.
QuerySurge multi machine installation
A) Install the QuerySurge Database Server
1. Download the QuerySurge Installer to the machine you want to install the QuerySurge Database Server on.
2. Double click the QuerySurge Installer, Under Setup click Next. Select "I accept the License Agreement" and click Next.
3. To accept the default Directory location click Next. If not specify desired Directory and click Next.
3. Under "Select Components", select "Database" and click Next.
4. To accept the default Memory Configuration click Next, if not select "Custom Memory Configuration" and insert adjusted values, click Next.
5. To accept default QuerySurge Database Port (3306) click Next, if not insert adjusted value and click Next.
Note: Make a note of the machine name and HTTP port number that you use; you will need it for the next part of the installation.
6. Under "Statsmon Connection" click Next.
7.An alert message will inform you that if you are upgrading from a QuerySurge version prior to 5.0.X, you will need to apply a licensee with the Staging Feature enabled before you can run Staging Queries. Click Next.
8. Click Finish.
B) Install the QuerySurge Application Server
1. Download the QuerySurge installer on the machine you want to download the QuerySurge App Server.
2. Double click the QuerySurge Installer. Under "Setup" click Next.
3. To accept the License Agreement select "I accept the License Agreement" and click Next.
4. If you want to accept the default Directory location click Next, or specify Directory location and click Next.
5.Under "Select Components" select "App Server" and click Next.
6.To accept the default Memory Configuration click Next, if not select "Custom Memory Configuration" and insert adjusted values, click Next.
7. If you applied for any add-on features like the BI-Tester insert your Feature Code you received via email now and click Next. If not click Next to continue.
8. To accept the default App Server HTTP Port and App Server Shutdown Port click Next. If not insert adjusted values and click Next.
9.Use the default QuerySurge Application Server Ports, unless either is already in use, and click the Next button to continue.Note: of the machine name and HTTP port number that you use; you will need it for the next part of the installation.
10. Enter the Hostname of the QuerySurge database server and the port you specified when you installed the database (see the previous section).
11. Click the Next to proceed to the installation of the QuerySurge Application Server.
C) Configure Statsmon for the Database
- Log into your Database machine if you are not still logged in.
- Launch the Windows Services console, and scroll down to the QuerySurgeStatsmon service.
- Stop the QuerySurgeStatsmon service by right-clicking on the service and selecting Stop in the context menu. The service may take a while to stop.
- Go to your QuerySurge install directory (the default is: C:\Program Files\QuerySurge). Open the following subdirectory: \\..\statsmon\config. If you installed in the default location, the full path is: C:\Program Files\QuerySurge\statsmon\config.
- Make a copy of the statsmonconfig.xml file found in this directory.
-
Carefully edit the statsmonconfig.xml file using Wordpad or other similar text editor. The tag you'll need to edit is <qsserverurl>. Carefully change the text value of the tag to the server name or IP address of your QuerySurge App server. After you have edited the tag, it will look something like:
<qsserverurl>http://myQuerySurgeAppserver.mycompany.com:80</qsserverurl>In this example, myQuerySurgeAppserver.mycompany.com is the name of the server you installed the QuerySurge App server on (the IP address of this server should also work), and 80 is the http port for the QuerySurge App server (see above Sec. B Step 4).
- Save your changes to the statsmonconfig.xml file.
- In the Windows Services console, start the QuerySurgeStatsmon service.
D) Install the QuerySurge Agent
1. Download the QuerySurge Installer to the box you want to install the QuerySurge Agent on.
2. Double click on the QuerySurge Installer to start the installation. To accept the License Agreement select "I accept the agreement" and click Next.
3. To accept the Default Directory click Next. If not specify Directory location and click Next.
4. Under "Select Components" be sure to select "Agent" Next.
5. To accept the default Memory Configuration click Next. if not, insert adjusted values and click Next.
6. Select Bundled Driver you would like QuerySurge to connect to. Click Next.Note: If you do not see your specific data source technology listed, you can add additional vendor drivers to QuerySurge post-installation. See Appendix A or your vendor website for driver download information.
7.You may have to accept the License Agreements for the selected Drivers. Select "I accept the agreement" to accept the License Agreements for the Drivers you installed. Click Next once you have accepted each required license.
8.Select a Bundled Driver and click Next.
10. Set the Agent connection to the QuerySurge Server by selecting the "Test Agent Connection With Server" checkbox. Enter the Hostname of the QuerySurge App Server and the HTTP port you specified when you installed the database (see the previous section). Click Next to test the connection.
9.Select Test Agent Connection With Server. Insert the QuerySurge App Server Hostname and the HTTP Port from the Database Installation and press Next.
10.Click Next to proceed to the installation of the QuerySurge Agent. Run the Agent installation on multiple Agent machines for multiple Agents.
11.Click Next to proceed.Note: The Agent will auto-register with QuerySurge as part of installation. You will need to log into QuerySurge as Administrator to Enable each installed Agent.
Note: The local firewall may require ports to be opened. The default QuerySurge ports are 80 (HTTP) and 3306 (database), however you may have modified these during installation. If the ports used during installation are blocked, the firewall may be the cause.
Logging into QuerySurge™
1.Open a browser and navigate to the URL: http://<server_name>/QuerySurge.
Note:The URL is: http://<server_name>:<port>/QuerySurge) if a HTTP Port other then "80" was specified.
2.Log in with default administrative credentials:
Username: admin
Password: admin
3.Navigate to the QuerySurge™Administration area. In the Administration Tree to the left click Agents. Select Enabled for every Agent you installed.
Note: The Administration area is also where you can reset your username and password
QuerySurge™ licensing
If you have downloaded a trial version of QuerySurge™, a 15-day trial license has been included as part of the installation process. You are ready to start your QuerySurge™ trial.
QuerySurge™ getting started
QuerySurge™ comes with a Getting Started assistant at the top menu to help you get going.It will walk you through tasks as a new user.
QuerySurge™ configuration
You are now ready to configure your QuerySurge™ installation with connections to your data sources and targets (additional Agents and Users are also configuration options).
Note: After installing QuerySurge be sure to utilize the Setup Wizard™ in the upper right hand corner toolbar before beginning the QuickStart™ Guide. If you have not created the necessary connections for both Source and Target, QuerySurge™ will not be able to access your data.
Important: Be sure to back up the QuerySurge database regularly in order to Safeguard QuerySurge Data. Your only protection against data loss is a regular backup schedule. In the case of a severe outage your only possibility of recovery is based on a recent backup.
See this Knowledge Base article for backup procedures.
Documentation
- System Requirements Guide – hardware and software needed to minimally and optimally run QuerySurge.
- Configuration Guides. (Most of these configurations are automated in the Setup Wizard within QuerySurge).
o Configuring Connections: IBM DB2
o Configuring Connections: Delimited Flat Files
o Configuring Connections: Fixed Width Flat Files
o Configuring Connections: Oracle
o Configuring Connections: Teradata
o Configuring Connections: Microsoft SQL Server
o Configuring Connections: Microsoft SQL Server with Windows Authentication
o Configuring Connections: IBM PureData System for Analysis (Netezza)
o Configuring Connections: Hadoop Hive
Appendix A – Vendor JDBC Drivers
The following is a list of vendor website URLs for common supported drivers that are not bundled with QuerySurge. These drivers can be used with QuerySurge after a manual install step. Please note that some of the download pages are not public and are restricted to customers with logins.
The drivers for the products listed below appear as options in the QuerySurge Connection Wizard. If your driver does not appear as an option in the QuerySurge Connection Wizard, it likely can be supported using the Generic option in the Connection Wizard.
Aster nClusterhttp://downloads.teradata.com/download/connectivity/jdbc-driver
IBM DB2 Universal Driver Type 4http://www-01.ibm.com/software/data/db2/
IBM Informix 11.7http://www14.software.ibm.com/webapp/download/search.jsp?go=y&rs=ifxjdbc
IBM Netezzahttp://www-01.ibm.com/software/data/netezza/
Sybase jConnector 7http://www.sybase.com/products/allproductsa-z/softwaredeveloperkit/jconnect