In Weaviate, how to get the total count of vector search results when using pagination?

In Weaviate, how to get the total count of vector search results when using pagination?

  

We are using nearText filter in our vector search with pagination using limit and offset - as can be seen below is the code snippet using Weaviate Java SDK Client:

    NearTextArgument nearTextArgument = weaviateClient.graphQL().arguments().nearTextArgBuilder()
            .concepts(new String[]{ "basketball" })
            .certainty(0.6f)
            .build();
    Result<GraphQLResponse> result2 = weaviateClient.graphQL().get()
            .withNearText(nearTextArgument)
            .withClassName(...)
            .withFields(...)
            .withLimit(...)
            .withOffset(...)
            .run();

However, we need to provide a feedback to our API if there is still a valid next page (e.g. page 3 for a size of 25 data) for the search results, as well as the total count of search hits which I could not find in the documentation.

I read that one can use the aggregate function however, it does not support withOffset

Is there a way to perform vector search with pagination using offset and include the total count of hits? And if yes, I would appreciate a sample code snippet on how to achieve these requirements.

Answer

I've just found a way of achieving this. I'm not sure whether this is the best approach, but here it is. Say you are executing the following GraphQL query (from here):

{
  Get {
    Article(where: {
      path: ["wordCount"],    # Path to the property that should be used
      operator: GreaterThan,  # operator
      valueInt: 1000          # value (which is always = to the type of the path property)
    }) {
      title
    }
  }
}

You would be able to count the results by adding Aggregate to the query with the same filters as Get:

{
  Aggregate {
    Article(where: {
      path: ["wordCount"],    # Path to the property that should be used
      operator: GreaterThan,  # operator
      valueInt: 1000          # value (which is always = to the type of the path property)
    }) {
      meta {
        count
      }
    }
  }
}
{
  Get {
    Article(where: {
      path: ["wordCount"],    # Path to the property that should be used
      operator: GreaterThan,  # operator
      valueInt: 1000          # value (which is always = to the type of the path property)
    }) {
      title
    }
  }
}

Then you would be able to get the total count in data.Aggregate.Article[0].meta.count.

© 2024 Dagalaxy. All rights reserved.