Package org.apache.lucene.util.hnsw
Class IncrementalHnswGraphMerger
java.lang.Object
org.apache.lucene.util.hnsw.IncrementalHnswGraphMerger
- All Implemented Interfaces:
HnswGraphMerger
- Direct Known Subclasses:
ConcurrentHnswMerger
This merges multiple graphs in a single thread in incremental fashion.
- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionprotected static final recordRepresents a vector reader that contains graph info. -
Field Summary
FieldsModifier and TypeFieldDescriptionprotected final intprotected final FieldInfoprotected List<IncrementalHnswGraphMerger.GraphReader> protected final intprotected final RandomVectorScorerSupplier -
Constructor Summary
ConstructorsConstructorDescriptionIncrementalHnswGraphMerger(FieldInfo fieldInfo, RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth) -
Method Summary
Modifier and TypeMethodDescriptionaddReader(KnnVectorsReader reader, MergeState.DocMap docMap, Bits liveDocs) Adds a reader to the graph merger if it meets the following criteria: 1.protected HnswBuildercreateBuilder(KnnVectorValues mergedVectorValues, int maxOrd) Builds a new HnswGraphBuilderprotected final int[][]getNewOrdMapping(KnnVectorValues mergedVectorValues, BitSet initializedNodes) merge(KnnVectorValues mergedVectorValues, InfoStream infoStream, int maxOrd) Merge and produce the on heap graph
-
Field Details
-
fieldInfo
-
scorerSupplier
-
M
protected final int M -
beamWidth
protected final int beamWidth -
graphReaders
-
-
Constructor Details
-
IncrementalHnswGraphMerger
public IncrementalHnswGraphMerger(FieldInfo fieldInfo, RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth) - Parameters:
fieldInfo- FieldInfo for the field being merged
-
-
Method Details
-
addReader
public IncrementalHnswGraphMerger addReader(KnnVectorsReader reader, MergeState.DocMap docMap, Bits liveDocs) throws IOException Adds a reader to the graph merger if it meets the following criteria: 1. does not contain any deleted docs 2. is a HnswGraphProvider/PerFieldKnnVectorReader- Specified by:
addReaderin interfaceHnswGraphMerger- Parameters:
reader- KnnVectorsReader to add to the mergerdocMap- MergeState.DocMap for the readerliveDocs- Bits representing live docs, can be null- Returns:
- this
- Throws:
IOException- If an error occurs while reading from the merge state
-
createBuilder
protected HnswBuilder createBuilder(KnnVectorValues mergedVectorValues, int maxOrd) throws IOException Builds a new HnswGraphBuilder- Parameters:
mergedVectorValues- vector values in the merged segmentmaxOrd- max num of vectors that will be merged into the graph- Returns:
- HnswGraphBuilder
- Throws:
IOException- If an error occurs while reading from the merge state
-
getNewOrdMapping
protected final int[][] getNewOrdMapping(KnnVectorValues mergedVectorValues, BitSet initializedNodes) throws IOException - Throws:
IOException
-
merge
public OnHeapHnswGraph merge(KnnVectorValues mergedVectorValues, InfoStream infoStream, int maxOrd) throws IOException Description copied from interface:HnswGraphMergerMerge and produce the on heap graph- Specified by:
mergein interfaceHnswGraphMerger- Parameters:
mergedVectorValues- view of the vectors in the merged segmentinfoStream- optional info stream to set to buildermaxOrd- max number of vectors that will be added to the graph- Returns:
- merged graph
- Throws:
IOException- during merge
-