Bag of Words databases

The visual search is performed using a technique called Bag of Words or Bag of Features.

Examples

Basic searching

The default visual database class is SiftColornamesWrapper, which is a convenience wrapper for making combined searches in a SIFT and color names database.

from vsearch.database import SiftColornamesWrapper

# Load the database
database = SiftColornamesWrapper.from_files('my_sift_db.h5',
                                            'my_cname_db.h5')

# Define a region of interest
roi = [x, y, width, height]

# Query by path to image
matches = database.query_path('some_image.jpg', roi, max_result=5)

# Query by image
image = ...
matches = database.query_image(image, roi, max_distance=0.8)

Matches are returned as a list of (key, distance) tuples, sorted by distance (ascending).

Dealing with location data

Adding location data to the database is done using the DatabaseWithLocation wrapper class. It uses a DatabaseEntry class which contains location in the form of LatLng objects for location data. Accessing entries is done like any dictionary/mapping:

from vsearch.database import SiftColornamesWrapper, DatabaseWithLocation

visualdatabase = SiftColornamesWrapper.from_files('my_sift_db.h5',
                                            'my_cname_db.h5')

locdatabase = DatabaseWithLocation(visualdatabase)
entry = locdatabase['some_key']
print('{} location is lat={:.6f}, lng={:.6f}'.format(entry.key, entry.latlng.lat, entry.latlng.lng))

We can also update the location:

new_location = LatLng(12.3456, 45.678)
locdatabase['some_key'] = new_location

Visual databases

class vsearch.database.SiftColornamesWrapper(sift_db, cname_db)

A database that combines a SIFT and color names databse

For each query, both the SIFT and color names database will be queried. The resulting matches are then sorted using the minimum of the SIFT and color names distance value.

combine_matches(sift_matches, cname_matches)

Combine SIFT and colornames matches

classmethod from_files(sift_db_path, cname_db_path)

Load the database from a SIFT and color names database

query_image(image, roi)

Query using an image array and region of interest

Parameters:
  • image (np.ndarray) – Image array
  • roi (array_like) – Region of interest encoded as [x, y, width, height]
Returns:
Return type:

Sorted list of database matches [(key1, distance1), (key2, distance2), ..] where distance1 < distance2.
query_path(path, roi)

Query using an image path and region of interest

Parameters:
  • path (str) – Path to the query image
  • roi (array_like) – Region of interest encoded as [x, y, width, height]
Returns:
Return type:

Sorted list of database matches [(key1, distance1), (key2, distance2), ..] where distance1 < distance2.
class vsearch.database.SiftFeatureDatabase(vocabulary)

An ANN database for SIFT features

query_image(image, roi)

Query using an image array and region of interest

Parameters:
  • image (np.ndarray) – Image array
  • roi (array_like) – Region of interest encoded as [x, y, width, height]
Returns:
Return type:

Sorted list of database matches [(key1, distance1), (key2, distance2), ..] where distance1 < distance2.
query_path(path, roi)

Query using an image path and region of interest

Parameters:
  • path (str) – Path to the query image
  • roi (array_like) – Region of interest encoded as [x, y, width, height]
Returns:
Return type:

Sorted list of database matches [(key1, distance1), (key2, distance2), ..] where distance1 < distance2.
class vsearch.database.ColornamesFeatureDatabase(vocabulary)

An ANN database for color names features

query_image(image, roi)

Query using an image array and region of interest

Parameters:
  • image (np.ndarray) – Image array
  • roi (array_like) – Region of interest encoded as [x, y, width, height]
Returns:
Return type:

Sorted list of database matches [(key1, distance1), (key2, distance2), ..] where distance1 < distance2.
query_path(path, roi)

Query using an image path and region of interest

Parameters:
  • path (str) – Path to the query image
  • roi (array_like) – Region of interest encoded as [x, y, width, height]
Returns:
Return type:

Sorted list of database matches [(key1, distance1), (key2, distance2), ..] where distance1 < distance2.

Location database

class vsearch.database.DatabaseWithLocation(visualdb)

A database that also contains location data

query_image(image, roi)

Query using an image array and region of interest

Parameters:
  • image (np.ndarray) – Image array
  • roi (array_like) – Region of interest encoded as [x, y, width, height]
Returns:
Return type:

Sorted list of database matches [(entry1, distance1), (entry2, distance2), ...] where distance1 < distance2 and entryX are DatabaseEntry instances.
query_path(path, roi)

Query using an image path and region of interest

Parameters:
  • path (str) – Path to the query image
  • roi (array_like) – Region of interest encoded as [x, y, width, height]
Returns:
Return type:

Sorted list of database matches [(entry1, distance1), (entry2, distance2), ...] where distance1 < distance2 and entryX are DatabaseEntry instances.

Types

class vsearch.database.DatabaseError

Error raised on database errors

class vsearch.database.LatLng(lat, lng)

Tuple for locations in the form of a latitude and longitude

lat

Latitude

lng

Longitude

class vsearch.database.DatabaseEntry(key, bow, latlng)

Tuple for database entries

key

The database key. Usually the filename.

bow

Bag of Words vector (raw word frequency)

latlng

Location in the form of a LatLng object.