Class FSIndexRepositoryImpl

java.lang.Object
org.apache.uima.cas.impl.FSIndexRepositoryImpl
All Implemented Interfaces:
FSIndexRepositoryMgr, FSIndexRepository, LowLevelIndexRepository

public class FSIndexRepositoryImpl extends Object implements FSIndexRepositoryMgr, LowLevelIndexRepository
There is one instance of this class per CAS View. Some parts of the data here are shared between all views of a CAS. Many things refer to specific types, and their associated Java Cover classes. There are 2 kinds of cover classes: If JCas is not being used, or if there is no JCas cover class defined for a type, then the Java class FeatureStructureImplC is used as the cover class. If the JCas is being used, then the JCas cover type (a subtype of TOP) is used as the cover class. Both of these classes inherit from FeatureStructureImpl (an abstract class) Both of these classes implement the common interface FeatureStructure. Generic typing: User facing APIs can make use of the (JCas) Java cover types, for indexes and iterators over them The general generic type used is typically written here as T extends FeatureStructure, where FeatureStructure is the super interface of all cover types (JCas and non-JCas). APIs having no reference to Java cover types (i.e., low level iterators) are not generic, unless they are needed to be to pass along the associated type to other APIs.
  • Field Details

    • DEBUG

      private static final boolean DEBUG
      See Also:
    • DEFAULT_INDEX_SIZE

      public static final int DEFAULT_INDEX_SIZE
      The default size of an index.
      See Also:
    • ALLOW_DUP_ADD_TO_INDEXES

      public static final String ALLOW_DUP_ADD_TO_INDEXES
      Define this JVM property to allow adding the same identical FS to Set and Sorted indexes more than once.
      See Also:
    • IS_ALLOW_DUP_ADD_2_INDEXES

      public static final boolean IS_ALLOW_DUP_ADD_2_INDEXES
    • DISABLE_ENHANCED_WRONG_INDEX

      public static final String DISABLE_ENHANCED_WRONG_INDEX
      See Also:
    • IS_DISABLE_ENHANCED_WRONG_INDEX_CHECK

      private static final boolean IS_DISABLE_ENHANCED_WRONG_INDEX_CHECK
    • emptyFSIterator

      private static final FSIterator emptyFSIterator
    • emptyLlIterator

      private static final LowLevelIterator emptyLlIterator
    • cas

      private final CASImpl cas
      Replicated per view
    • locked

      private boolean locked
    • indexArray

      An array of ArrayLists, one for each type in the type hierarchy. The ArrayLists are unordered lists of IndexIteratorCachePairs for that type, corresponding to the different index definitions over that type. This list includes iicps for subtypes of defined and default indexes over some type
    • detectIllegalIndexUpdates

      final int[] detectIllegalIndexUpdates
      an array of ints, one for each type in the type hierarchy. Used to enable iterators to detect modifications (adds / removes) to indexes they're iterating over while they're iterating over them. Not private so it can be seen by FSLeafIndexImpl
    • name2indexMap

      A map from names to IndexIteratorCachePairs, which represent the index at the top-most type declared in the index specification. Different names may map to the same iicp. The keys are the same across all views, but the values are different, per view
    • indexUpdates

      private final IntVector indexUpdates
    • indexUpdateOperation

      private final BitSet indexUpdateOperation
    • logProcessed

      private boolean logProcessed
    • usedIndexes

      private final IntVector usedIndexes
    • isUsed

      private final boolean[] isUsed
    • iteratedSortedIndexes

      private final List<FSIndexRepositoryImpl.IndexIteratorCachePair<? extends FeatureStructure>> iteratedSortedIndexes
    • sii

    • mPii

  • Constructor Details

    • FSIndexRepositoryImpl

      private FSIndexRepositoryImpl()
      ----------------------- Support for flattened indexes -----------------
    • FSIndexRepositoryImpl

      FSIndexRepositoryImpl(CASImpl cas)
      Constructor. Assumption: called first before next constructor call, with the base CAS view
      Parameters:
      cas -
    • FSIndexRepositoryImpl

      FSIndexRepositoryImpl(CASImpl cas, FSIndexRepositoryImpl baseIndexRepo)
      Constructor for views.
      Parameters:
      cas -
      baseIndexRepository -
  • Method Details

    • createPointerIterator

      Create an "ordered" (e.g. one that maintains iicp sort order for sorted index) pointer iterator over an iicp
      Parameters:
      iicp - - the index plus its subtype list of indexes
      Returns:
      an int iterator
    • createPointerIterator

      <T extends FeatureStructure> IntPointerIterator createPointerIterator(FSIndexRepositoryImpl.IndexIteratorCachePair<? extends FeatureStructure> iicp, boolean is_unordered)
      Create an ordered or iicp-unordered pointer iterator over an iicp Note that flattened index iterators are not int style; and they're created higher up...
      Parameters:
      iicp - - the index plus its subtype list of indexes
      is_unordered - true if ordering among subtypes not needed
      Returns:
      an int iterator
    • createPointerIterator

      <T extends FeatureStructure> IntPointerIterator createPointerIterator(FSIndexRepositoryImpl.IndexIteratorCachePair<? extends FeatureStructure> iicp, int fs)
    • createPointerIterator

      <T extends FeatureStructure> IntPointerIterator createPointerIterator(FSIndexRepositoryImpl.IndexIteratorCachePair<? extends FeatureStructure> iicp, boolean is_unordered, int fs)
    • createLeafPointerIterator

      private <T extends FeatureStructure> IntPointerIterator createLeafPointerIterator(FSIndexRepositoryImpl.IndexIteratorCachePair<? extends FeatureStructure> iicp)
    • getIndexesForType

      <T extends FeatureStructure> ArrayList<FSIndexRepositoryImpl.IndexIteratorCachePair<T>> getIndexesForType(int typeCode)
    • init

      private void init()
      Initialize data. Called from the constructor.
    • resetDetectIllegalIndexUpdates

      private void resetDetectIllegalIndexUpdates()
    • flush

      public void flush()
      Reset all indexes, in one view.
    • clearIteratedSortedIndexes

      private void clearIteratedSortedIndexes()
    • addFS

      public void addFS(int fsRef)
    • addNewIndex

      private FSIndexRepositoryImpl.IndexIteratorCachePair<? extends FeatureStructure> addNewIndex(FSIndexComparator comparator, int indexType)
    • addNewIndex

      private <T extends FeatureStructure> FSIndexRepositoryImpl.IndexIteratorCachePair<T> addNewIndex(FSIndexComparator comparator, int initialSize, int indexType)
      This is where the actual index gets created.
    • addNewIndexCore

      private <T extends FeatureStructure> FSLeafIndexImpl<T> addNewIndexCore(FSIndexComparator comparator, int initialSize, int indexType)
    • isAnnotationIndex

      private boolean isAnnotationIndex(Type type, FSIndexComparator comp)
    • addNewIndexRecursive

      private FSIndexRepositoryImpl.IndexIteratorCachePair<? extends FeatureStructure> addNewIndexRecursive(FSIndexComparator comparator, int indexType)
      Top level call to add the indexes for a particular index definition
      Parameters:
      comparator -
      indexType -
      Returns:
      the iicp for the top new index
    • findIndex

      private static final <T extends FeatureStructure> int findIndex(ArrayList<FSIndexRepositoryImpl.IndexIteratorCachePair<T>> indexes, FSIndexComparator comp, int indexType)
      Finds an index among iicp's for all defined indexes of a type, such that the type of the index (SET, BAG, SORTED) is the same and the comparator (the keys) are the same
      Parameters:
      indexes -
      comp -
      indexType -
      Returns:
      the index in the set of iicps for this type for the matching index
    • addNewIndexRec

      private FSIndexRepositoryImpl.IndexIteratorCachePair<? extends FeatureStructure> addNewIndexRec(FSIndexComparator comparator, int indexType)
      Add an index for a type, and then (unless it's a DEFAULT_BAG_INDEX), call yourself recursively to add the indexes for all the directly subsumed subtypes.
      Parameters:
      comparator -
      indexType -
      Returns:
      the new iicp for the new index
    • getAllSubsumedTypes

      private static final ArrayList<Type> getAllSubsumedTypes(Type t, TypeSystem ts)
    • addAllSubsumedTypes

      private static final void addAllSubsumedTypes(Type t, TypeSystem ts, ArrayList<Type> v)
    • commit

      public void commit()
      Description copied from interface: FSIndexRepositoryMgr
      Commit this repository instance. No more additions will be allowed.
      Specified by:
      commit in interface FSIndexRepositoryMgr
      See Also:
    • getDefaultTypeOrder

      public LinearTypeOrder getDefaultTypeOrder()
      Description copied from interface: FSIndexRepositoryMgr
      Get the default type order.
      Specified by:
      getDefaultTypeOrder in interface FSIndexRepositoryMgr
      Returns:
      The default type order.
    • getDefaultOrderBuilder

      public LinearTypeOrderBuilder getDefaultOrderBuilder()
      Description copied from interface: FSIndexRepositoryMgr
      Get the default type order builder.
      Specified by:
      getDefaultOrderBuilder in interface FSIndexRepositoryMgr
      Returns:
      The default type order builder.
    • setDefaultTypeOrder

      void setDefaultTypeOrder(LinearTypeOrder order)
    • createIndex

      public boolean createIndex(FSIndexComparator comp, String label, int indexType) throws CASAdminException
      Description copied from interface: FSIndexRepositoryMgr
      Create a new index. Note: if you creata a BAG_INDEX, the comparator will be ignored.
      Specified by:
      createIndex in interface FSIndexRepositoryMgr
      Parameters:
      comp - The comparator for the new index.
      label - The name of the new index.
      indexType - The kind of index (sorted, set, bag).
      Returns:
      false iff an index with the samelabel already exists.
      Throws:
      CASAdminException - If the repository is locked (after calling commit()).
      See Also:
    • createIndexNoQuestionsAsked

      public boolean createIndexNoQuestionsAsked(FSIndexComparator comp, String label, int indexType)
      This is public only until the xml specifier format supports specifying index kinds (set, bag etc.).
      Parameters:
      comp - -
      label - -
      indexType - -
      Returns:
      -
    • getIndexes

      public Iterator<FSIndex<FeatureStructure>> getIndexes()
      Description copied from interface: FSIndexRepository
      Get all indexes in this repository.
      Specified by:
      getIndexes in interface FSIndexRepository
      Returns:
      All indexes.
      See Also:
    • ll_getIndexes

      public Iterator<LowLevelIndex> ll_getIndexes()
      Description copied from interface: FSIndexRepository
      Get all indexes in this repository as low level indexes
      Specified by:
      ll_getIndexes in interface FSIndexRepository
      Returns:
      All indexes.
    • getLabels

      public Iterator<String> getLabels()
      Description copied from interface: FSIndexRepository
      Get all labels for all indexes.
      Specified by:
      getLabels in interface FSIndexRepository
      Returns:
      All labels.
      See Also:
    • getLabels

      public Iterator<String> getLabels(FSIndexComparator comp)
      Get the labels for a specific comparator.
      Parameters:
      comp - The comparator.
      Returns:
      An iterator over the labels.
    • getIndex

      public <T extends FeatureStructure> FSIndex<T> getIndex(String label, Type type)
      Description copied from interface: FSIndexRepository
      Retrieve an index according to a label and a type. The type is used to narrow down the index of a more general type to a more specific one. Generics: T is the associated Java cover class for the type.
      Specified by:
      getIndex in interface FSIndexRepository
      Type Parameters:
      T - The Java class associated with the type
      Parameters:
      label - The name of the index.
      type - A subtype of the type of the index.
      Returns:
      The specified, or null if an index with that name doesn't exist.
      See Also:
    • getIndex

      public <T extends FeatureStructure> FSIndex<T> getIndex(String label)
      Description copied from interface: FSIndexRepository
      Retrieve an index according to a label. Generics: T is the associated Java cover class for the top type defined for this index name.
      Specified by:
      getIndex in interface FSIndexRepository
      Type Parameters:
      T - the Java class associated with the top-most type of the index
      Parameters:
      label - The name of the index.
      Returns:
      The index with the name label, or null if no such index is defined.
      See Also:
    • getIntIteratorForIndex

      public IntPointerIterator getIntIteratorForIndex(String label)
    • getIntIteratorForIndex

      public IntPointerIterator getIntIteratorForIndex(String label, Type type)
    • getIndexSize

      public int getIndexSize(Type type)
    • removeAllExcludingSubtypes

      public void removeAllExcludingSubtypes(Type type)
      Remove all instances of a particular type (but not its subtypes) from all indexes
      Specified by:
      removeAllExcludingSubtypes in interface FSIndexRepository
      Parameters:
      type - -
    • removeAllIncludingSubtypes

      public void removeAllIncludingSubtypes(Type type)
      Remove all instances of a particular type (including its subtypes) from all indexes
      Specified by:
      removeAllIncludingSubtypes in interface FSIndexRepository
      Parameters:
      type - -
    • createComparator

      public FSIndexComparator createComparator()
      Description copied from interface: FSIndexRepositoryMgr
      Create a new comparator to define a new index.
      Specified by:
      createComparator in interface FSIndexRepositoryMgr
      Returns:
      A new comparator instance.
      See Also:
    • isCommitted

      public boolean isCommitted()
      Description copied from interface: FSIndexRepositoryMgr
      Check if this instance has been committed.
      Specified by:
      isCommitted in interface FSIndexRepositoryMgr
      Returns:
      true iff this instance has been committed.
      See Also:
    • createIndex

      public boolean createIndex(FSIndexComparator comp, String label) throws CASAdminException
      Description copied from interface: FSIndexRepositoryMgr
      Create a new sorted index.
      Specified by:
      createIndex in interface FSIndexRepositoryMgr
      Parameters:
      comp - The comparator for the new index.
      label - The name of the new index.
      Returns:
      false iff an index with the samelabel already exists.
      Throws:
      CASAdminException - If the repository is locked (after calling commit()).
      See Also:
    • getIndexedFSs

      public int[] getIndexedFSs()
      For one particular view (the one associated with this instance of FsIndexRepositoryImpl), return an array containing all FSs in any defined index, in this view. This is intended to be used for serialization. Note that duplicate entries are removed. It used to be that the items were sorted by fs addr, but that only happens if the dedup code runs The order in which FSs occur in the array does not reflect the order in which they were added to the repository. This means that set indexes deserialized from this list may contain different but equal elements than the original index.
      Returns:
      an array containing all FSs in any defined index, in this view.
    • addFS

      public void addFS(FeatureStructure fs)
      Description copied from interface: FSIndexRepository
      Add a feature structure to all appropriate indexes in the repository. If no indexes exist for the type of FS that you are adding, then a bag (unsorted) index will be automatically created.

      Important: after you have called addFS() on a FS, do not change the values of any features used for indexing. If you do, the index will become corrupted and may be unusable. If you need to change an index feature value, first call removeFS() on the FS, change the feature values, then call addFS() again.

      Specified by:
      addFS in interface FSIndexRepository
      Parameters:
      fs - The FS to be added.
      See Also:
    • incrementIllegalIndexUpdateDetector

      private void incrementIllegalIndexUpdateDetector(int typeCode)
    • removeFS

      public void removeFS(FeatureStructure fs)
      Description copied from interface: FSIndexRepository
      Remove a feature structure from all indexes in the repository.
      Specified by:
      removeFS in interface FSIndexRepository
      Parameters:
      fs - The FS to be removed.
      See Also:
    • removeFS

      public void removeFS(int fsRef)
    • createTypeSortOrder

      public LinearTypeOrderBuilder createTypeSortOrder()
      Description copied from interface: FSIndexRepositoryMgr
      Currently not useful.
      Specified by:
      createTypeSortOrder in interface FSIndexRepositoryMgr
      Returns:
      A new type order builder.
    • ll_getIndex

      public LowLevelIndex ll_getIndex(String indexName)
      Description copied from interface: LowLevelIndexRepository
      Get an index by its name.
      Specified by:
      ll_getIndex in interface LowLevelIndexRepository
      Parameters:
      indexName - The name of the index.
      Returns:
      The corresponding index, or null if no such index exists.
    • ll_getIndex

      public LowLevelIndex ll_getIndex(String indexName, int typeCode)
      Description copied from interface: LowLevelIndexRepository
      Get an index by a name and type. The type must be a subtype of the index's type. Note that there is no special checked version of this method, the type parameters are always checked.
      Specified by:
      ll_getIndex in interface LowLevelIndexRepository
      Parameters:
      indexName - The name of the index.
      typeCode - The code of the desired subtype.
      Returns:
      The corresponding index, or null if no such index exists.
    • ll_addFS

      public final void ll_addFS(int fsRef, boolean doChecks)
      Description copied from interface: LowLevelIndexRepository
      Add a FS reference to all appropriate indexes in the repository.
      Specified by:
      ll_addFS in interface LowLevelIndexRepository
      Parameters:
      fsRef - The FS reference to be added to the repository. If fsRef is not a valid FS reference, the subsequent behavior of the system is undefined.
      doChecks - Check if the FS reference argument is a valid reference.
    • ll_addFS

      public void ll_addFS(int fsRef)
      Description copied from interface: LowLevelIndexRepository
      Add a FS reference to all appropriate indexes in the repository.
      Specified by:
      ll_addFS in interface LowLevelIndexRepository
      Parameters:
      fsRef - The FS reference to be added to the repository. If fsRef is not a valid FS reference, the subsequent behavior of the system is undefined.
    • ll_addback

      public void ll_addback(int fsRef, int count)
    • ll_addFS_common

      private void ll_addFS_common(int fsRef, boolean isAddback, int count)
    • getAutoIndexNameForType

      private static final String getAutoIndexNameForType(Type type)
    • ll_removeFS_ret

      boolean ll_removeFS_ret(int fsRef)
    • ll_removeFS_all_ret

      int ll_removeFS_all_ret(int fsRef)
      Remove potentially multiple times from all defined indexes for this view
      Parameters:
      fsRef - the item to remove
      Returns:
      the number of times the FS was added to the index, may be 0
    • ll_removeFS

      public void ll_removeFS(int fsRef)
      Description copied from interface: LowLevelIndexRepository
      Remove a FS reference from all indexes in the repository. Note that this only removes the reference from the index repository, it does not free memory on the heap.
      Specified by:
      ll_removeFS in interface LowLevelIndexRepository
      Parameters:
      fsRef - The FS reference to be removed from the indexes.
    • ll_getAllIndexedFS

      public LowLevelIterator ll_getAllIndexedFS(Type type)
    • ll_getAllIndexedFS

      private final void ll_getAllIndexedFS(Type type, List<LowLevelIterator> iteratorList)
    • getAllIndexedFS

      public <T extends FeatureStructure> FSIterator<T> getAllIndexedFS(Type type)
      Description copied from interface: FSIndexRepository
      Gets an iterator over all indexed FeatureStructures of the specified Type (and any of its subtypes). The elements are returned in arbitrary order, and duplicates (if they exist) are not removed. Generics: T is the Java class for aType.
      Specified by:
      getAllIndexedFS in interface FSIndexRepository
      Type Parameters:
      T - The Java class associated with aType
      Parameters:
      type - The type
      Returns:
      An iterator that returns all indexed FeatureStructures of type aType and its subtypes, in no particular order.
    • getAllIndexedFS

      private final <T extends FeatureStructure> void getAllIndexedFS(Type type, List<FSIteratorImplBase<T>> iteratorList)
    • addDirectSubtypes

      private <T extends FeatureStructure> void addDirectSubtypes(Type type, List<FSIteratorImplBase<T>> iteratorList)
    • ll_addDirectSubtypes

      private void ll_addDirectSubtypes(Type type, List<LowLevelIterator> iteratorList)
    • isInSetOrSortedIndexInThisView

      public boolean isInSetOrSortedIndexInThisView(int fsAddr)
      This is used to see if a FS which has a key feature being modified could corrupt an index in this view. It returns true if found (sometimes it returns true, even if strictly speaking, there is no chance of corruption - see below) It does this by seeing if this FS is indexed by one or more Set or Sorted indexes. If found in the first Sorted index encountered, return true Else, if found in any Set index, return true, otherwise false To speed up the 2nd case, when there are more than one Set indexes, we do an approximation if there are any bag indexes: we check if found in the first bag index encountered -- return true if found This is an approximation in that it could be that none of the Set indexes contain that FS, but it is in the bag index. So, this method can sometimes return true incorrectly. If type is subtype of annotation, can just test the built-in Annotation index.
      Parameters:
      fsAddr - the FS to see if it is in some index that could be corrupted by a key feature value change
      Returns:
      true if this fs is found in a Set or Sorted index.
    • removeIfInCorrputableIndexInThisView

      int removeIfInCorrputableIndexInThisView(int fsAddr)
      This is used when deserializing a FS using delta CAS which could be modifying an existing one (below the line). In order to update if the FS is in any index (in any view) and it has new key values for some key used in the indexes, it has to be removed, updated, and then readded back to the indexes The current implementation does not try to determine if any keys are being updated, it just assumes one or more are. Optimization: If type is subtype of annotation, can just test the built-in Annotation index. If the view has nothing other than bag indexes for this type, return false without doing any remove
      Parameters:
      fsRef - - the FS to see if it is in some index that could be corrupted by a key feature value change
      Returns:
      count of number of instances of this fs that were removed in this view, if the view had one or more Set or Sorted indexes, otherwise 0
    • ll_removeFS_all

      int ll_removeFS_all(int fsAddr)
      Remove this fsAddr from all defined indexes in this view for this type
      Parameters:
      fsAddr - - the item to remove all instances of
      Returns:
      the number of times this FS was added to the indexes, may be 0 if not in any index
    • getAnnotationIndexNoSubtypes

      private <T extends FeatureStructure> FSIntArrayIndex<T> getAnnotationIndexNoSubtypes(int typeCode)
      returns the annotation index for a type which is Annotation or a subtype of it.
      Parameters:
      typeCode -
      Returns:
      the index just for that type
    • logIndexOperation

      private void logIndexOperation(int fsRef, boolean added)
    • processIndexUpdates

      private void processIndexUpdates()
      Go through the journal, and use those entries to update added, deleted, and reindexed lists in such a way as to guarantee: a FS is in only one of these lists, (or in none) For a journal "add-to-indexes" event: fs in "deleted": remove from "deleted", add to "reindexed" fs in "reindexed": do nothing fs in "added": do nothing fs not in any of these: add to "added" For a journal "remove-from-indexes" event: fs in "added": remove from "added" (don't add to "deleted") fs in "reindexed": remove from "reindexed" and add to "deleted") fs in "deleted": do nothing fs not in any of these: add to "deleted" The journal is cleared after processing.
    • getUpdatedFSs

      public int[] getUpdatedFSs(PositiveIntSet items)
    • getAddedFSs

      public int[] getAddedFSs()
    • getDeletedFSs

      public int[] getDeletedFSs()
    • getReindexedFSs

      public int[] getReindexedFSs()
    • isModified

      public boolean isModified()
    • toString

      public String toString()
      Overrides:
      toString in class Object
    • getAnnotationFsComparator

      Comparator<AnnotationFS> getAnnotationFsComparator()
    • getAnnotationIntComparator

      IntComparator getAnnotationIntComparator()