Notice: Undefined offset: in /my/Zend/ib/Search/Lucene/Index/SegmentInfo.php on line 641-Collection of common programming errors

2 years ago
admin
4 minutes

I’ve a problem in indexing with Zend Search Lucene framework.

Our file repository has ~25000 files and I’m trying to index them. But during batch indexing, this error occurred:

Notice: 
Undefined offset: 2047 in /my/Zend/lib/Search/Lucene/Index/SegmentInfo.php on line 641 
Notice: 
Trying to get property of non-object in /my/Zend/lib/Search/Lucene/Index/SegmentMerger.php on line 202 
Fatal error: 
Uncaught exception 'Zend_Search_Lucene_Exception' with message 'Error occured while file reading.' in /my/Zend/lib/Search/Lucene/Storage/File/Filesystem.php:174 
Stack trace: 
#0 /my/Zend/lib/Search/Lucene/Storage/File.php(470): Zend_Search_Lucene_Storage_File_Filesystem->_fread(524287) 
#1 /my/Zend/lib/Search/Lucene/Index/SegmentMerger.php(203): Zend_Search_Lucene_Storage_File->readBinary() 
#2 /my/Zend/lib/Search/Lucene/Index/SegmentMerger.php(126): Zend_Search_Lucene_Index_SegmentMerger->_mergeStoredFields() 
#3 /my/Zend/lib/Search/Lucene/Index/Writer.php(385): Zend_Search_Lucene_Index_SegmentMerger->merge() 
#4 /my/Zend/lib/Search/Lucene/Index/Writer.php(341): Zend_Search_Lucene_Index_Writer->_mergeSegments(Array) 
#5 /my/Zend/lib/Search/Lucene/Index/Writer.php(250): Zend_Search_Lucene_Index_Writer->_maybeMergeSegments() 
#6 /my/Zend/lib/Search/Luc in /my/Zend/lib/Search/Lucene/Storage/File/Filesystem.php on line 174

It occured when my program index 1000th file.

my code is here:

(I send fileNames through an ajax code.)

$handle = fopen('/my/index/directory/my.lock', 'w');
if (flock($handle, LOCK_EX)){
    $mergeFactor = 300;
    $fileName = $_POST['fileName'];
    $index = Zend_Search_Lucene::open($path);
    $doc = new FileDocument($fileName,true);
    $similarity = new Search_Similarity();
    Zend_Search_Lucene_Search_Similarity::setDefault($similarity);
    Zend_Search_Lucene_Search_QueryParser::setDefaultEncoding('UTF-8');
    Zend_Search_Lucene_Analysis_Analyzer::setDefault(new Zend_Search_Lucene_Analysis_Analyzer_Common_Utf8_CaseInsensitive());
    $index->setMergeFactor($mergeFactor);
    //if ($index->count() % 50 == 0){
    //  $index->optimize();
    //}
    $index->addDocument($doc);
    flock($handle, LOCK_UN);
}else {
    echo "UNABLE To Lock!";
}
fclose($handle);

My Zend Version: 1.11.11

and when I toggle comments to optimize my index manually, this error occured in another file:

Notice: 
Undefined offset: 16383 in /my/Zend/lib/Search/Lucene/Index/SegmentInfo.php on line 641 
Notice: 
Trying to get property of non-object in /my/Zend/lib/Search/Lucene/Index/SegmentMerger.php on line 202 
Fatal error: 
Uncaught exception 'Zend_Search_Lucene_Exception' with message 'Error occured while file reading.' in /my/Zend/lib/Search/Lucene/Storage/File/Filesystem.php:174 
Stack trace: 
#0 /my/Zend/lib/Search/Lucene/Storage/File.php(470): Zend_Search_Lucene_Storage_File_Filesystem->_fread(81919) 
#1 /my/Zend/lib/Search/Lucene/Index/SegmentMerger.php(203): Zend_Search_Lucene_Storage_File->readBinary() 
#2 /my/Zend/lib/Search/Lucene/Index/SegmentMerger.php(126): Zend_Search_Lucene_Index_SegmentMerger->_mergeStoredFields() 
#3 /my/Zend/lib/Search/Lucene/Index/Writer.php(385): Zend_Search_Lucene_Index_SegmentMerger->merge() 
#4 /my/Zend/lib/Search/Lucene/Index/Writer.php(807): Zend_Search_Lucene_Index_Writer->_mergeSegments(Array) 
#5 /my/Zend/lib/Search/Lucene.php(1456): Zend_Search_Lucene_Index_Writer->optimize() 
#6 /my/Zend/lib/Search/Lucene/Proxy.php(518): Zend in /my/Zend/lib/Search/Lucene/Storage/File/Filesystem.php on line 174

  1. What’s my problem?!

  2. What’s my solution?! 😀

Thanks. 🙂

  1. I have the same problem, but I have to add some lines of code in Zend Lucene, becauce couldn’t find solution. There are two checking on existing field, below:

    in file: Zend/Search/Lucene/Index/SegmentInfo.php

    public function getField($fieldNum)
{
    if (isset($this->_fields[$fieldNum])) // add this line to check field in the fields array
        return $this->_fields[$fieldNum];
}

    in file: Zend/Search/Lucene/Index/SegmentManager.php

    private function _mergeStoredFields()
{
    //..
    foreach ($this->_segmentInfos as $segName => $segmentInfo) {
        // ..
        for ($count = 0; $count < $segmentInfo->count(); $count++) {
            // ..
            for ($count2 = 0; $count2 < $fieldCount; $count2++) {
                // ..
                if (!$fieldInfo)    // add this lines
                    continue;       // to exclude empty field
            }
            // ..
      }
      // ..
    }
    // ..
}

    Better late than never :)))

    UPD: hm.. unfortunately as I learn this code isn’t a solution 🙁

Originally posted 2013-11-09 23:40:14.