3225157 - /hana/data & hana/log volumes consumption reaches critical size post re-partitioning operations on a large table

• A large table is (re)partitioned on a HANA Database. For example : 
   
   

    alter table PCL2
    partition by range(<column_name>) 
    (partition '000'<= VALUES<'<>',
    partition '<>'<= VALUES <'<>',
    partition others);

• Before the partitioning was carried out, the '/hana/data' volume had 348GB free, as below.
   
   

    sidadm@host:/home/...>  df -h | grep data
    /dev/mapper/hanadatavg-<>      1.8T  1.5T  348G  82% /hana/data

• The partition job is successful. When the command finished, /hana/data reaches 93% as below. /hana/log is at 46%.
   
   

    /dev/mapper/hanadatavg-<>     1.8T  1.7T  146G  93% /hana/data
    /dev/mapper/hanalogvg-<>        512G  232G  281G  46% /hana/log

• However minutes later, /hana/data reaches 100% as below. /hana/log was still at 46% as all changes mad been recorded.
   
   

   /dev/mapper/hanadatavg-<>      1.8T  1.8T  1.6M 100% /hana/data
   /dev/mapper/hanalogvg-<>        512G  232G  281G  46% /hana/log

• indexserver traces show diskfull events toward the end and just after the partition job finishes :
   
   

    [62853]{-1}[-1/-1] 2022-07-08 14:43:17.259852 i Savepoint        SavepointImpl.cpp(03044) : Savepoint current savepoint version: 411217, restart redo log position: 0x65aaab79d0, next savepoint version: 411218, last snapshot SP version: 0
    [54947]{-1}[-1/-1] 2022-07-08 14:48:44.170342 i EventHandler     LocalFileCallback.cpp(00495) : [DISKFULL] (1st request) [W] , buffer= 0x0001ff6b82330000, offset= 1970802786304, size= 0/16777216, file= "<root>/datavolume_0000.dat" ((open, mode= RW, access= rw-rw-r--, flags= ASYNC|DIRECT|MUST_NOT_EXIST|LOCK), factory= (root= "/hana/data/<SID> /mnt00001/hdb00003.00003/" (access= rw-rw-r--, flags= AUTOCREATE_PATH, usage= DATA, fs= xfs, config= (async_write_submit_active=on,async_write_submit_blocks=all,async_read_submit=on,num_submit_queues=1,num_completion_queues=1,size_kernel_io_queue=512,max_parallel_io_requests=64,min_submit_batch_size=16,max_submit_batch_size=64))) {shortRetries= 0, fullRetries= 0 (0/10)}
    [54947]{-1}[-1/-1] 2022-07-08 14:48:44.170390 i EventHandler     EventManagerImpl.cpp(00675) : New event reported: 'DiskFullEvent[id= 2, path= /hana/data/<SID>/mnt00001/hdb00003.00003/, state= NEW]'
    [54947]{-1}[-1/-1] 2022-07-08 14:48:44.170399 i Logger           LoggerImpl.cpp(00104) : Logger notified of new DiskFull: DiskFullEvent[id= 2, path= /hana/data/<SID> /mnt00001/hdb00003.00003/, state= NEW]
  
    . . . . . .
  
    [14240]{-1}[-1/-1] 2022-07-08 16:02:18.278810 i EventHandler     EventManagerImpl.cpp(00951) : Event 'DiskFullEvent[id= 2, path= /hana/data/<SID>/mnt00001/hdb00003.00003/, state= HANDLING]' set to state 'handled'

M_SAVEPOINT statistics show very large pageflushes as part of the savepoint writes occurring as expected, both during and after the partition operation has finished :

In all :

242.83 GB flushed to disk @ 14:48 pm

45.8 GB flushed to disk @ 14:43 pm

150.73 GB flushed to disk @ 12:30 pm 

This means that ~ 439.3 GB worth of modified pages were flushed and written to the 'hana/data' disk in a space of just 20 minutes.

SAP HANA, platform edition 2.0

SAP HANA, platform edition 1.0

partitioning, pageflush, hana/data, hana/log, re-partition, diskfull , KBA , HAN-DB-PER , SAP HANA Database Persistence , Problem