Vector CQL data type

A new CQL data type, a VECTOR is required for Vector Search. issues.apache.org/jira/browse/CASSANDRA-18504 added this data type to Cassandra 5.0. The new type VECTOR has the following properties:

fixed length array
elements may not be null
flatten array (aka multi-cell = false)

Although the data type specified for a vector used in vector search is a floating point number, the vector data type can be used to create a vector of any data type. For example, a vector of int or bigint can be created, but cannot be used with vector search.

Vectors are limited to a maximum dimension of 8K (2^13) items.

Create a keyspace:

CREATE KEYSPACE IF NOT EXISTS cycling
   WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : '1' };

Use the keyspace:

USE cycling;

This data type can be used in a CREATE TABLE statement:

CREATE TABLE IF NOT EXISTS cycling.comments_vs (
  record_id timeuuid,
  id uuid,
  commenter text,
  comment text,
  comment_vector VECTOR <FLOAT, 5>,
  created_at timestamp,
  PRIMARY KEY (id, created_at)
)
WITH CLUSTERING ORDER BY (created_at DESC);

1	Create a 5-dimensional embedding

Index the vector column:

CREATE INDEX IF NOT EXISTS ann_index
  ON cycling.comments_vs(comment_vector) USING 'sai';

Insert data:

INSERT INTO cycling.comments_vs (record_id, id, created_at, comment, commenter, comment_vector)
   VALUES (
      now(),
      e7ae5cf3-d358-4d99-b900-85902fda9bb0,
      '2017-02-14 12:43:20-0800',
      'Raining too hard should have postponed',
      'Alex',
      [0.45, 0.09, 0.01, 0.2, 0.11]
);
INSERT INTO cycling.comments_vs (record_id, id, created_at, comment, commenter, comment_vector)
   VALUES (
      now(),
      e7ae5cf3-d358-4d99-b900-85902fda9bb0,
      '2017-03-21 13:11:09.999-0800',
      'Second rest stop was out of water',
      'Alex',
      [0.99, 0.5, 0.99, 0.1, 0.34]
);
INSERT INTO cycling.comments_vs (record_id, id, created_at, comment, commenter, comment_vector)
   VALUES (
      now(),
      e7ae5cf3-d358-4d99-b900-85902fda9bb0,
      '2017-04-01 06:33:02.16-0800',
      'LATE RIDERS SHOULD NOT DELAY THE START',
      'Alex',
      [0.9, 0.54, 0.12, 0.1, 0.95]
);

INSERT INTO cycling.comments_vs (record_id, id, created_at, comment, commenter, comment_vector)
   VALUES (
      now(),
      c7fceba0-c141-4207-9494-a29f9809de6f,
      totimestamp(now()),
      'The gift certificate for winning was the best',
      'Amy',
      [0.13, 0.8, 0.35, 0.17, 0.03]
);

INSERT INTO cycling.comments_vs (record_id, id, created_at, comment, commenter, comment_vector)
   VALUES (
      now(),
      c7fceba0-c141-4207-9494-a29f9809de6f,
      '2017-02-17 12:43:20.234+0400',
      'Glad you ran the race in the rain',
      'Amy',
      [0.3, 0.34, 0.2, 0.78, 0.25]
);

INSERT INTO cycling.comments_vs (record_id, id, created_at, comment, commenter, comment_vector)
   VALUES (
      now(),
      c7fceba0-c141-4207-9494-a29f9809de6f,
      '2017-03-22 5:16:59.001+0400',
      'Great snacks at all reststops',
      'Amy',
      [0.1, 0.4, 0.1, 0.52, 0.09]
);
INSERT INTO cycling.comments_vs (record_id, id, created_at, comment, commenter, comment_vector)
   VALUES (
      now(),
      c7fceba0-c141-4207-9494-a29f9809de6f,
      '2017-04-01 17:43:08.030+0400',
      'Last climb was a killer',
      'Amy',
      [0.3, 0.75, 0.2, 0.2, 0.5]
);

Select data:

SELECT * FROM cycling.comments_vs
    ORDER BY comment_vector ANN OF [0.15, 0.1, 0.1, 0.35, 0.55]
    LIMIT 3;

Cassandra Documentation

Version:

Vector CQL data type