3237673 - HANA log volume segments filling up in a "Truncated" state due to a missing/misconfigured backint agent

• The log segments are observed to be filling up the 'hana/log' volume rapidly.

• A check of the log segments via M_LOG_SEGMENTS or 'hdblogdiag seglist' reveals most , if not all, are in a "Truncated" state
  

  
  sidadm@hostname:/usr/sap/SID/SYS/global/hdb/custom/config> hdblogdiag seglist /hana/log/SID/mnt00001/hdb00001 | grep -i Truncated | wc -l
  
  870

• indexserver traces and the backint + backup.log files reveal continuous log backup failures :

    [82262]{-1}[-1/-1] 2022-08-15 14:59:48.148226 e Backup           BackupExe_LogBackup.cpp(00882) : Report backup failure: exception  1: no.110503  (unknown file:0) TID: 82262
       Backint cannot execute /usr/sap/SID/SYS/global/hdb/opt/hdbbackint, No such file or directory (2)
       [82262]{-1}[-1/-1] 2022-08-15 15:00:48.229668 e Logger           BackupHandlerImpl.cpp(00477) : Error during log segment backup: exception  1: no.110503  (unknown file:0) TID: 82262
       Backint cannot execute /usr/sap/SID/SYS/global/hdb/opt/hdbbackint, No such file or directory (2)
       exception throw location:
        1: 0x00007f6ca23fcf47 in Backup::BackupDestBackint_ExecutorDaemon::executeImpl(Backup::BackupDestBackint_Job::JobType, unsigned long, unsigned long, Backup::BackupDestBackint_Job::LevelType, char const*, char const*, char const*, char const*, char const*, int&)+0x593 at BackupDestBackint_ExecutorDaemon.cpp:71 (libhdbbackup.so)
        2: 0x00007f6ca24099fc in Backup::BackupDestBackint_Executor::execute(Backup::BackupDestBackint_Job&)+0x558 at BackupDestBackint_Executor.cpp:164 (libhdbbackup.so)
        3: 0x00007f6ca2495564 in Backup::BackupExe_BackintHandlerImpl::startBackintForBackup(ltt::vector<ltt::shared_ptr<Backup::BackupDest_Destination, ltt::default_deleter, ltt::integral_constant<bool, false>, ltt::integral_constant<bool, false> > > const&, unsigned long, unsigned long, Backup::BackupDestBackint_Job::LevelType, Backup::BackupDestBackint_Job::BackupType, ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> > const&, ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> > const&, BackupTools::ToolsCom_Version const&, ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> > const&, unsigned long)+0x540 at BackupExe_BackintHandlerImpl.cpp:213 (libhdbbackup.so)
        4: 0x00007f6ca249385e in Backup::BackupExe_BackintHandlerImpl::startBackintForBackup(ltt::vector<ltt::shared_ptr<Backup::BackupDest_Destination, ltt::default_deleter, ltt::integral_constant<bool, false>, ltt::integral_constant<bool, false> > > const&, unsigned long, unsigned long, Backup::BackupDestBackint_Job::LevelType, Backup::BackupDestBackint_Job::BackupType, ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> > const&, ltt::basic_string<char, ltt::char_traits<char>, ltt::integral_constant<int, 64> > const&, BackupTools::ToolsCom_Version const&, unsigned long)+0x8a at BackupExe_BackintHandlerImpl.cpp:92 (libhdbbackup.so)
        5: 0x00007f6ca246f90b in Backup::BackupExe_LogBackupImpl::LogBackupThread::collectionBackupStartBackint(ltt::auto_ptr<Backup::BackupExe_LogWaitForBackintConnectionThread, ltt::default_deleter, ltt::integral_constant<bool, false> >&, ltt::vector<ltt::shared_ptr<Backup::BackupDest_Destination, ltt::default_deleter, ltt::integral_constant<bool, false>, ltt::integral_constant<bool, false> > >&, unsigned long)+0x1c7 at BackupExe_LogBackup.cpp:316 (libhdbbackup.so)
        6: 0x00007f6ca24736b3 in Backup::BackupExe_LogBackupImpl::LogBackupThread::executeCollectionBackup(ltt::smart_ptr<Backup::BackupExe_LogBackupInformationCollection, ltt::integral_constant<bool, false>, ltt::integral_constant<bool, false> >&)+0x210 at BackupExe_LogBackup.cpp:548 (libhdbbackup.so)
        7: 0x00007f6ca247099e in Backup::BackupExe_LogBackupImpl::LogBackupThread::run(void*&)+0x57a at BackupExe_LogBackup.cpp:830 (libhdbbackup.so)
        8: 0x00007f6c977f13fd in Execution::Thread::staticMainImp(Execution::Thread*)+0x529 at Thread.cpp:522 (libhdbbasis.so)
        9: 0x00007f6c977e2b39 in Execution::pthreadFunctionWrapper(Execution::Thread*)+0x165 at ThreadInterposition.cpp:639 (libhdbbasis.so)
       10: 0x00007f6c96e604f9 in start_thread+0xd5 (libpthread.so.0)
       11: 0x00007f6c9553aecf in __GI___clone+0x3b (libc.so.6)
       exception type information:
        - 0: public ltt::exception@0x7f6c96dc6c70
        
         . . . . . 
          
       [82262]{-1}[-1/-1] 2022-08-15 15:52:53.143082 e Backup           BackupExe_LogBackup.cpp(00882) : Report backup failure: exception  1: no.110503  (unknown file:0) TID: 82262
           Backint cannot execute /usr/sap/SID/SYS/global/hdb/opt/hdbbackint, No such file or directory (2)
       [82261]{-1}[-1/-1] 2022-08-15 15:53:15.336134 i Savepoint        SavepointImpl.cpp(03049) : Savepoint current savepoint version: 140031, restart redo log position: 0x7c95a3e582, next savepoint version: 140032, last snapshot SP version: 140028
       [82262]{-1}[-1/-1] 2022-08-15 15:53:53.228223 e Logger           BackupHandlerImpl.cpp(00477) : Error during log segment backup: exception  1: no.110503  (unknown file:0) TID: 82262
           Backint cannot execute /usr/sap/SID/SYS/global/hdb/opt/hdbbackint, No such file or directory (2)
       exception throw location:
       
        . . . . . 
       
       
       [82262]{-1}[-1/-1] 2022-08-15 15:53:53.233546 e Backup           BackupExe_LogBackup.cpp(00882) : Report backup failure: exception  1: no.110503  (unknown file:0) TID: 82262
           Backint cannot execute /usr/sap/SID/SYS/global/hdb/opt/hdbbackint, No such file or directory (2)

SAP HANA, platform edition 2.0

SAP HANA, platform edition 1.0

Truncated, hana/log, M_LOG_SEGMENTS, hdblogdiag seglist, BackupExe_LogBackup.cpp, BackupHandlerImpl.cpp , KBA , HAN-DB-PER , SAP HANA Database Persistence , HAN-DB-BAC , SAP HANA Backup & Recovery , Problem