Configuring the Search Engine

Magnify can use either the Lucene search engine, which is built into Magnify, or the Google Search Appliance (GSA), which is a separately purchased product. In either case, WebFOCUS must know the location of the search engine, the XSLT style sheet to use for the Magnify user interface, and the maximum number of items that should appear in the Dynamic Categorization tree. (These items have preset defaults.)

For more information on configuring Magnify to use the Google Search Appliance, see Technical Memo 4699: Using Magnify With The Google Search Engine.

The following procedure explains how to set the search engine parameters using the WebFOCUS Administration Console.

Procedure: How to Set Search Engine Parameters in the WebFOCUS Console

To set the search engine parameters in WebFOCUS:

In the left pane of the WebFOCUS Administration Console, select Magnify, then General Settings.
The following image shows the WebFOCUS Administration Console with the default Client Settings for Magnify in the right pane.
From the magnify_search_engine drop-down list, select the search engine you want to configure with Magnify. The selections are:
- Built-In Lucene. (The Lucene search engine is included with Magnify.)
- Google Search Appliance. (You must purchase this product separately.)
  For more information on configuring Magnify to use the Google Search Appliance, see Technical Memo 4699:Using Magnify With The Google Search Engine.
Type values (or accept the default values) for the remaining parameters:
- magnify_search_engine_url. URL to the Google Search Appliance. This is only active for the Google Search Appliance (GSA).
- magnify_root. Full path to the directory where all Magnify configuration files are located, including the lucene index, cached index files, log files, and the Magnify style sheet.
  
  The default path is
```
&IBI_DOCUMENT_ROOT/magnify
```
- magnify_config. Full path to the directory where the configuration files are located.
  
  If your installation needs to make changes to the configuration directory, you should move this directory outside of the ibi directory. This ensures that future WebFOCUS upgrades do not overwrite your configuration information.
  
  The default path is
```
&IBI_DOCUMENT_ROOT/config/magnify
```
- magnify_stylesheet_dir. Full path to the directory where the Magnify style sheets reside. This setting enables you to set the path to a directory outside of your WebFOCUS installation to prevent it from being overwritten when you perform an upgrade or a reinstallation.
  
  The default is
```
&IBI_DOCUMENT_ROOT/config/magnify
```
  Important: Be sure to include a slash at the end of the path to the style sheet directory.
- magnify_lucene_index_dir. Full path to the directory where the default Lucene index resides. This is only active for the Lucene search engine.
  
  The default directory is
```
&IBI_DOCUMENT_ROOT/magnify/lucene_index
```
  Additional search directories are configured in the following file:
```
config/magnify/collections.xml
```
- magnify_recordlimit. Maximum number of URL items to be returned by a search request. (These items will appear in the Dynamic Categorization tree.) Any results beyond this number are not displayed to the user. The default value is 300.
- magnify_feed_cache_dir. Directory where Magnify stores records which are not yet loaded into the index. The files stored in this directory are files which are generated from the iWay Service Manager feed process. After all content is added to the Lucene index, the cached version is moved or deleted depending on the magnify_generate_deltas variable value of true or false. An index during a quiecse is loaded into the index after the backup of the Lucene index is complete.
  
  The default value is
```
&IBI_DOCUMENT_ROOT/magnify/feedcache
```
- magnify_required_cookie. If a HTTP cookie name is specified, Magnify verifies that the cookie is set. If the required cookie does not exist, Magnify displays the Web page specified by the magnify_required_cookie_redirect parameter. If the parameter is set to false (the default value), Magnify does not verify any cookies.
- magnify_required_cookie_redirect. Web page that Magnify displays if the HTTP cookie specified by the magnify_required_cookie parameter does not exist. By default, the WebFOCUS Welcome page displays.
Click Save.

Configuration of the search engine for Magnify is complete.

Setting Timers For Feeding Data

When indexing large amounts of records, the information is not available for search until it is committed to the index. Magnify enables administrators to set timers for when prolonged indexing occurs. Commits can be issued during this time, thereby updating the index library. This ensures that the latest version of the index library is available when a search is performed.

Magnify timers regulate Magnify operations while transmitting feeds to the index library. This affects how and when new search content is made available to the Magnify search-based application. This is useful when indexing large amounts of data. Timers control Magnify operations such as open, close, and write. This helps tune the Magnify platform for various indexing activities by adjusting times to help control the frequency at which Magnify operations take place, thereby assisting in memory and performance allocation.

When Magnify receives incoming feeds, they are first held in memory for processing, then they are written to the physical index library. Once a feed is written, then it can be made available to the Magnify search-based application. After the search syncs its view of the index library, the newly fed record will be returned as a search result.

There are several different timers that can be configured using the settings in the following page:

http://server_name:port_number/context_root/search/jsp/setIndexWriterTimers.jsp

Alternatively, you can access the timer settings by clicking Magnify Timers in the Magnify Console.

The following page displays.

Magnify Timer Settings

You can set the following timers for the index:

Flush Timer. Controls the length of time to hold an incoming stream of Magnify feeds in memory before being written to the index library. Larger volumes of data should be given more time and memory. This results in less I/O usage, but it increases the latency of the feeds written to the disk. Therefore, higher times are recommended for historic, first-time, or other large batch indexing processing. For large index streams, it is recommended that the flush timer is set to 600 or more seconds and for incremental/real-time loads at 60 seconds.
Commit Timer. Controls the length of time to wait at the end of an incoming stream of Magnify feeds before being written to the index library. The commit timer setting must be less than or equal to the Flush timer setting to avoid flushing empty memory to the index library. This is activated as soon as there is a pause or gap in the incoming stream to Magnify. Usually, this value should be the same as the Flush Timer.
Note: When indexing, the Magnify Administrator can set the Commit Timer to a very low value (for example, 5 seconds) so that Magnify users retrieve the new search data almost immediately.
Close Timer. Sets the length of time after the last incoming document has been processed before performing closing operations on the index library. This results in disk I/O operations and once complete, the write.lock file is removed. With higher times, feeds that may result in small breaks between incoming streams, are not required to use resources on open operations. Also, with bulk indexing, higher times to improve index times is recommended, but lessens the time to open and close indexes. The default value is 2 minutes.
Inactivity Timer. Specifies the frequency in which to check the duration of the close and commit timer cycles. This is recommended to be the same as the close timer setting. This setting requires a very small set of I/O. Only in the most extreme cases is this setting different than the close timer setting.
Reading Refresh Rate. Controls the intervals at which the Magnify search application refreshes its cached view of the Magnify index library. This is applicable when multiple application servers are used to feed and search Magnify. For example, when a single WeFOCUS environment is created with two application servers. One application server is used for WebFOCUS reporting and Magnify searching, the other application server is used for indexing. Each application server is configured with different port numbers. To sync up the searching and indexing processes, the Magnify readers must be refreshed.

Important: Changes made to the timer settings should be tested before being applied in a production environment, since data size, memory allocations, and CPU specifications differ between machines.