Gaffer, a spy software used by UK Government has been open sourced.

If you have a github account, you can fork out the project from below link:

https://github.com/GovernmentCommunicationsHeadquarters/Gaffer/

 

Gaffer is essentially a framework for processing large amounts of interconnected data, and build graphs based on that data. This in turn can be used to analyse the relationships between those objects.

(AN = alphanumeric)

For indexing purposes, Oracle Commerce divides characters into 3 categories:

  • AN characters
  • Non-AN characters (as configured in special chars config)
  • Other non-AN chars not configures in above.

Case folding
Oracle commerce search ops are not case sensitive, so alpha chars are always included in lowercase form at the time of indexing.

Indexing of non-AN chars
The non-AN chars which are not specified as searchable in config are treated depending upon whether they are considered punctuation or symbols.

  • Punctuation chars are treated as white space. The following are punctuation chars:
    ! @ # & ( ) - [ { } ] : ; ‘ , ? / *
    In a multi word search with words separated by punctuation, the word order is preserved as if it was a phrase search.
  • Symbols are also treated as white space, but word order is not preserved. The following are symbol characters
    ` ~ $ ^ + = < > "

(AN = alphanumeric)

For indexing purposes, Oracle Commerce divides characters into 3 categories:

  • AN characters
  • Non-AN characters (as configured in special chars config)
  • Other non-AN chars not configures in above.

Case folding
Oracle commerce search ops are not case sensitive, so alpha chars are always included in lowercase form at the time of indexing.

Indexing of non-AN chars
The non-AN chars which are not specified as searchable in config are treated depending upon whether they are considered punctuation or symbols.

  • Punctuation chars are treated as white space. The following are punctuation chars:
    ! @ # & ( ) - [ { } ] : ; ‘ , ? / *
    In a multi word search with words separated by punctuation, the word order is preserved as if it was a phrase search.
  • Symbols are also treated as white space, but word order is not preserved.
    If a symbol character is next to a punctuation character, it is ignored. 
    The following are symbol characters
    ` ~ $ ^ + = < > "

Oracle Commerce Search supports following search modes:

 

  • MatchAll
    MatchAll means all keywords specified by user should be present in the record. e.g. If a user searched for brown bag, the result should have both brown AND bag in the text.
  • MatchPartial
    Matches partial keywords e.g. brown bag will return all records which have EITHER brown OR bag in its properties. In addition, the settings specified in ‘match at least’ and ‘omit at most’ parameters, also must be satisfied.
  • MatchAny
    Matches any of the terms in the search query string.
  • MatchAllAny
    The engine first tries to match ALL, and if no results are found, it falls back to Match any.
  • MatchAllPartial
    First, the engine tries to find results for MatchAll. If no results are found, then MatchPartial is used.
  • MatchPartialMax
    First, the engine tries to find results for MatchAll. If no results exist, then the search is executed with one less term than original, and so on, until results are found. This mode is subject to Match at least and Omit at most.
  • MatchBoolean
    Allows users to specify complex expressions.

    ——————————————————————-

Ntx
Query parameter to specify match mode e..g Ntx=mode+matchall

Dx
Query parameter to use in dimension search

——————————————————————-

Types of dimension search

  • Default dimension search
  • Compound dimension search

What is compound dimension search ?
Compound dimension search enables MDEX engine to return combinations of dimension values, i.e. dimension states, that match a search query.

This search reduces to a default dimension search for single-term searches i.e. it is only relevant for multi-term searches.Compound dimension search results must satisfy following

  • Validity – Must have records
  • Coverage – Each dimension value MUST match at least 1 record
  • Minimalism – Removing one should remove results.

By default all dimensions are enabled for dimension search.

—compoundDimSearch
Use thisDgidx flag to enable dimensions for compound dimension search.

Use ENEQuery.setDimSearchCompound() to enable compound dimension search on a search request.

Dimension values within a dimension are ordered based on

  • Static dimension value ranks, OR
  • Relevance ranking

Ordering of compound dimension search results is based upon

  • No of dimensions within each result set
  • Secondary sort which is lexicographical
  • Values are sorted as per the static value ranks or relevancy ranking.

By default, dimension search returns dimensions with no records. To avoid this, use one of the below:

  • Dn=0 (Query param)
  • ENEQuery.setDimSearchNavDescriptors(DimValIdList(0))

 

Di
This query param can be used to limit default dimension search to a single dimension.

D=*
This query param can be used to return all possible dimension values across all dimensions.

Drc=id+11+numresults+1
When numresults is used in conjunction with Drc, it limits the number of dimension values to return.

Do
o for Offset. Use for paginating through Dimension search results.

Dk
Ranking for dimension values. Dk=0 or Dk=1. Not applicable to compound dimension search.Dn
Dimension search scope parameter. This searches within the results for dimension search.

Accessing dimension search results

[code language=”java”]ENEQueryResults.containsDimensionSearch
ENEQueryResults.DimenionSearch
[/code]


Enable Refinement counts for dimension search
This is done on a per query basis. Use showcounts with Drc query parameter.

Retrieving refinement counts
Inside PropertyMap on DimVal object, the value is stored against key ‘Dgraph.Bins’ (or ‘Dgraph.AggrBins’).

Performance impact of refinement count
Adding refinement count for dimension search adds an overhead, especially for rapidly executing typeahead searches. Be careful.