In Weaviate, how to get the total count of vector search results when using pagination?
We are using nearText
filter in our vector search with pagination using limit
and offset
- as can be seen below is the code snippet using Weaviate Java SDK Client:
NearTextArgument nearTextArgument = weaviateClient.graphQL().arguments().nearTextArgBuilder()
.concepts(new String[]{ "basketball" })
.certainty(0.6f)
.build();
Result<GraphQLResponse> result2 = weaviateClient.graphQL().get()
.withNearText(nearTextArgument)
.withClassName(...)
.withFields(...)
.withLimit(...)
.withOffset(...)
.run();
However, we need to provide a feedback to our API if there is still a valid next page
(e.g. page 3 for a size of 25 data) for the search results, as well as the total count of search hits
which I could not find in the documentation.
I read that one can use the aggregate
function however, it does not support withOffset
Is there a way to perform vector search with pagination using offset and include the total count of hits? And if yes, I would appreciate a sample code snippet on how to achieve these requirements.
Answer
I've just found a way of achieving this. I'm not sure whether this is the best approach, but here it is. Say you are executing the following GraphQL query (from here):
{
Get {
Article(where: {
path: ["wordCount"], # Path to the property that should be used
operator: GreaterThan, # operator
valueInt: 1000 # value (which is always = to the type of the path property)
}) {
title
}
}
}
You would be able to count the results by adding Aggregate
to the query with the same filters as Get
:
{
Aggregate {
Article(where: {
path: ["wordCount"], # Path to the property that should be used
operator: GreaterThan, # operator
valueInt: 1000 # value (which is always = to the type of the path property)
}) {
meta {
count
}
}
}
}
{
Get {
Article(where: {
path: ["wordCount"], # Path to the property that should be used
operator: GreaterThan, # operator
valueInt: 1000 # value (which is always = to the type of the path property)
}) {
title
}
}
}
Then you would be able to get the total count in data.Aggregate.Article[0].meta.count
.