Indexer Enhancements for 2.0
document description
Author : Bogdan Gheorghe
Revision Date : 12/08/2003 - Version: 0.1.0
Change History : 0.1.0 - Document Creation

Table of Contents

1. Introduction
2. Requirements
2. Lazy Evaluation
3. Priority Queue
4. Error Tasks
5. Build Info Changes
6. Scanner Configurations
7. Dependency Issues
8. Initial Index
9. Partial Results Query
10. Job Optimization
11. References


1. Introduction

The Indexer service was redone for the CDT 1.2 release and supports the Managed Make, Search, Navigation, and Code Assist features. Certain deficiencies have been identified through real world use of the Indexer supported features. This doc will address the main requirements on the Indexer for CDT 2.0.

2. Requirements

The additional requirements for the Indexer are captured in the table below:

ID
 Requirement
 Priority
R1
 Indexer must be able to finish all tasks started (no more lazy evaluation)
 P1
R2
 Indexer must be able to return dependency requests in a top priority manner P1
R3
 Indexer must be able to pass on information from the parser about encountered problems P1
R4
 Indexer should be activated when build settings change P2
R5
 Indexer should be activated to do a full reindex upon a config/indexer profile change P2
R6
 Dependencies should be calculated regardless of order of file creation
 P1
R7
 Initial Index should be done upfront at the start
 P3
R8
 Index should be able to answer a query at all times
 P1
R9
 Job Scheduling should be optimized
P2



3. Task Completion (R1)

Problem

The current indexing model is completely event driven - until an event is received to kick off the indexing, the indexer remains silent. This usually works out since an event is generated for all project creation/import actions as well as for all doc changes. It is possible, however, to abruptly shut down the workbench halfway through an indexing job. For example, a user decides to import a project and for whatever reason decides to quit the app/kick the power cord/smash the computer etc. This leaves the project with a partial index. The next time the user opens the workbench the indexer remains quiet until the user tries to use search or some other indexer supported feature. Now the indexer gets kicked again and has to reindex the entire project again. We could have been finishing the interrupted job in the background if we had known about it.

Requirement

The Indexer must be able to finish all tasks started, even those interrupted by catastrophic events.

3. Priority Queue (R2)

Problem

Managed Make depends on the indexer for its dependency generation. Currently, the dependency calculation is done as part of the indexing of files. The problem with this is that you can get into situations where the user imports an existing project into a new managed make project and wishes to build the project. The user is now forced to wait for the entire project to be indexed.

Requirement

The Indexer must be able to return dependency requests in a top priority manner.

4. Error Tasks (R3)

Problem

There is no elegant way of notifying the user if the parser runs into problems with scanner configuration items (i.e. can't find file, missing symbol definition). As a result, the user is in the dark when trying to figure out why parser related features aren't working properly.

Requirement

The Indexer must be able to pass on information from the parser about encountered problems.
5. Build Info Changes (R4)

Problem

There is no indexer trigger for build info changes. So even if the user tries to rectify a missing file problem for the scanner, the indexer won't be kicked until the file that includes the missing file is altered and saved. On the other hand, we don't want to reindex entire projects each time the user adds a new path or symbol.

Requirement

The Indexer should be activated when build settings change.

6. Scanner Config Changes (R5)

Problem

Managed Make can have more than one configurations defined. If the user switches from debug to release modes, new include paths and symbols become active. Currently, the indexer does not react to configuration changes. (This addresses the indexer profile enhancement request as well - each configuration can be considered as a separate Indexer profile).

Requirement

The Indexer should be activated to do a full reindex upon a config change.

 
6. Dependency Ordering (R6)

Problem

The user can create files in any order. It has been seen that if the user creates a source file before the header file, the header file won't get picked up by the indexer unless the source file is re-saved.

Requirement

The Indexer should be able to properly index files regardless of their order of creation.
 
7. Initial Index (R7)

Problem

When importing a project the indexer kicks off in the background - if the user wants to use search/navigation he must wait for the project to complete indexing.

Requirement

The Indexer should attempt to perform an index (cancelable) up front at project creation. This will allow the indexer to run at a higher priority, let the user know what's going on and ensure that when the user gets into the code the index will be ready to go. This feature can be turned on and off from a preference page.

 
8. Partial Results Query (R8)

Problem

All clients of the indexer need to wait for the indexer to successfully complete the index before allowing for queries.

Requirement

The Indexer should be able to halt indexing at anytime and perform a query based on whatever is indexed at that point.


9. Job Optimization (R9)

Problem

If a change is made to a file and saved an indexing job is created. If another change is made and the file is saved again a second index job is created despite the fact that the first job has not been processed yet.

Requirement

The Indexer must avoid scheduling duplicates of jobs.

10. References

CDT source code

Last Modified on Friday, January 16, 2004