2839881 - signal 11 in SSL_CTX_new - SAP ASE

• Job Scheduler exits or terminates with a signal 11 and a core dump
• SAP ASE Backupserver may also hang and backups teminate
• Corresponding error in the ASE errorlog is a stacktrace - 

00:0002:00000:00017:2020/11/10 12:55:16.73 kernel  Current process (0x37001c) infected with signal 11 (SIGSEGV)
00:0002:00000:00017:2020/11/10 12:55:16.73 kernel  Current Process is running on Engine 0
00:0002:00000:00017:2020/11/10 12:55:16.73 kernel  server is using elf symbols for stack decoding (125500 symbols found)
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel  Address 0x0x0000000004a4c8f8 (SSL_CTX_new), siginfo (code, address) = (2, 0x0x00007ff3bc4c7c40)
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel  **** Saved signal context (0x0x000000014eab5340): ****
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel  uc_flags: 0x7, uc_link: 0x(nil)
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel  uc_sigmask: 0x7bfbf037 0xb 0x2 0xbc4c7c40
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel  uc_stack: ss_sp: 0x(nil), ss_size: 0x0, ss_flags: 0x2
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel  General Registers (uc_mcontext.gregs):
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel       PC : 0x0000000004a4c8f8 (SSL_CTX_new)
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel          RAX : 0x00007ff3bc4c7c40  RBX : 0x00007ff3bcb46f00
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel          RCX : 0xfffffffffffff120  RDX : 0x0000000000000002
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel       RBP : (nil)  RSP : 0x000000014eab58f8
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel       R8  : (nil)  R9  : 0x00007ff3b800fd30
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel       R10 : 0x00000000a922c68e  R11 : 0x0000000004a4c8f8
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel       R12 : (nil)  R13 : 0x00007ff3bcb47280
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel       R14 : 0x00007ff3b800b320  R15 : (nil)
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel       RDI : 0x00007ff3bc4c7c40  RSI : 0x00007ff3bcb47310
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel       RIP : 0x0000000004a4c8f8  CSGSFS : 0x002b000000000033
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel       TRAPNO : 0x000000000000000e  ERR : 0x0000000000000007
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel       EFL : 0x0000000000010202
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel  **** end of signal context ****
00:0002:00000:00017:2020/11/10 12:55:17.35 kernel  ************************************
00:0002:00000:00017:2020/11/10 12:55:17.36 kernel  SQL causing error :
00:0002:00000:00017:2020/11/10 12:55:17.36 kernel  ************************************
00:0002:00000:00017:2020/11/10 12:55:17.36 server  SQL Text:
00:0002:00000:00017:2020/11/10 12:55:17.36 kernel  curdb = 31515 tempdb = 2 pstat = 0x10000 p2stat = 0x100100
00:0002:00000:00017:2020/11/10 12:55:17.36 kernel  p3stat = 0x800 p4stat = 0x0 p5stat = 0x0 p6stat = 0x1 p7stat = 0x10000
00:0002:00000:00017:2020/11/10 12:55:17.36 kernel  lasterror = 0 preverror = 0 transtate = 1
00:0002:00000:00017:2020/11/10 12:55:17.36 kernel  curcmd = 0 program =
00:0002:00000:00017:2020/11/10 12:55:17.36 kernel  extended error information: hostname:  login:
00:0002:00000:00017:2020/11/10 12:55:17.36 kernel  Job Scheduler Task found running by terminate_process, setting it to stopped
00:0002:00000:00017:2020/11/10 12:55:17.36 kernel  Job Scheduler Task final cleanup performed from terminate_process.
00:0002:00000:00017:2020/11/10 12:55:17.37 kernel  pc: 0x000000000154dc90 pcstkwalk+0x482()
00:0002:00000:00017:2020/11/10 12:55:17.37 kernel  pc: 0x000000000154d64f ucstkgentrace+0x20f()
00:0002:00000:00017:2020/11/10 12:55:17.37 kernel  pc: 0x0000000001549dd2 ucbacktrace+0x54()
00:0002:00000:00017:2020/11/10 12:55:17.37 kernel  pc: 0x00000000018198be terminate_process+0xb1e()
00:0002:00000:00017:2020/11/10 12:55:17.37 kernel  end of stack trace, spid 17, kpid 3604508, suid 0

• Corresponding errors in the JSAGENT.log could be found:

worker JS Agent hit signal 11
worker Core dump generated at /sybase/ABC/ASE-16_0/install/core.jsa_20190826_233635.22674
worker jsd_FreeContext: ct_exit() failed with retcode -4. Potential memory leak.
worker JS Agent exiting with signal 11

• SAP Adaptive Server Enterprise 16.0
• SAP Adaptive Server Enterprise 16.0 for Business Suite

SSL_CTX_new, Job Scheduler, backupserver, signal 11 , KBA , BC-DB-SYB , Business Suite on Adaptive Server Enterprise , Problem