19c trace文件暴增导致u01目录使用率过高

2025年7月2日 作者 XiaofeiHuangfu

适用范围
数据库-Oracle Database 19.12
操作系统-Cent OS 7.9
问题概述
Oracle 19c RAC 节点1在应用跑批时,trace目录不断增大,同时导致/u01目录使用率接近100%,影响数据库的正常使用。
问题原因
应用跑批package调用的对象不存在,SQL语句解析时产生大量trace。
解决方案
1、应用确认调用的对象,并重建不存在的对象。
2、应用重新跑批后。
分析过程
1、检查目录使用率

[oracle@XFPDB1 ~]$ df -h
Filesystem         Size  Used Avail Use% Mounted on
devtmpfs           126G     0  126G   0% /dev
tmpfs           s   126G   12G  114G  10% /dev/shm
tmpfs              126G  4.1G  122G   4% /run
tmpfs              126G     0  126G   0% /sys/fs/cgroup
/dev/sdb5          188G  119G   70G  64% /
/dev/sdb2          4.0G  137M  3.9G   4% /boot
/dev/sdb1          4.0G   12M  4.0G   1% /boot/efi
/dev/sdb3          220G  218G   2G  99% /u01
tmpfs               26G     0   26G   0% /run/user/1002
tmpfs               26G     0   26G   0% /run/user/1008
tmpfs               26G     0   26G   0% /run/user/1009
tmpfs               26G     0   26G   0% /run/user/0

检查发现/u01目录使用率已达到99%并在持续增长。
2、检查trace文件
找到最大的trace文件是xfpdb1_ora_23630.trc
查看trace内容

[oracle@XFPDB1 trace]$ more xfpdb1_ora_23630.trc
Trace file /u01/app/oracle/19C/diag/rdbms/xfpdb/xfpdb1/trace/xfpdb1_ora_23630.trc
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.12.1.0.0
Build label:    RDBMS_19.12.0.0.0DBRUR_LINUX.X64_211012
ORACLE_HOME:    /u01/app/oracle/19.0.0
System name:    Linux
Node name:      XFPDB1
Release:        3.10.0-1160.el7.x86_64
Version:        #1 SMP Mon Oct 19 16:18:59 UTC 2020
Machine:        x86_64
Instance name: xfpdb1
Redo thread mounted by this instance: 1
Oracle process number: 784
Unix process pid: 23630, image: oracle@XFPDB1


*** 2025-04-24T16:19:38.271795+08:00
*** SESSION ID:(1275.34219) 2025-04-24T16:19:38.271823+08:00
*** CLIENT ID:() 2025-04-24T16:19:38.271828+08:00
*** SERVICE NAME:(XFPDB) 2025-04-24T16:19:38.271833+08:00
*** MODULE NAME:(PL/SQL Developer) 2025-04-24T16:19:38.271838+08:00
*** ACTION NAME:(script HFXF.ETL_CNT@/XFPDB) 2025-04-24T16:19:38.271843+08:00
*** CLIENT DRIVER:() 2025-04-24T16:19:38.271848+08:00
 
KGI Error-3 = 9
Could not acquire mutex...Returning without dumping handle 0x39e333a08
KGI Error-3 = 9
Could not acquire mutex...Returning without dumping handle 0x39e333a08
KGI Error-3 = 9

LibraryHandle:  Address=0x39e333a08 Hash=48e92e5e LockMode=N PinMode=S LoadLockMode=0 Status=VALD Subpool=1 
  ObjectName:  Name=HFXF.PKG_XF_RS   
    FullHashValue=d42ae7cee549c4efb90f041348e92e5e Namespace=TABLE/PROCEDURE(01) Type=PACKAGE(09) ContainerId=0 ContainerUid=0 Identifier=2667149 OwnerI
dn=329 
  Statistics:  InvalidationCount=0 ExecutionCount=0 LoadCount=1 ActiveLocks=2 TotalLockCount=3 TotalPinCount=3 
  Counters:  BrokenCount=1 RevocablePointer=1 KeepDependency=0 Version=0 BucketInUse=0 HandleInUse=0 HandleReferenceCount=0 
  Concurrency:  DependencyMutex=0x39e333ab8(0, 15779415, 0, 0) Mutex=0x39e333b58(1275, 15779518, 92, 6) 
  Flags=PIN/TIM/[00002801] Flags2=[0000] 
  WaitersLists:  
    Lock=0x39e333a98[0x39e333a98,0x39e333a98] 
    Pin=0x39e333a78[0x39e333a78,0x39e333a78] 
    LoadLock=0x39e333af0[0x39e333af0,0x39e333af0] 
  Timestamp:  Current=05-15-2018 07:00:05 
  HandleReference:  Address=0x39e333c00 Handle=0x8afd36fa0 Flags=OWN[200] 
  LockInstance:  id='LBd42ae7cee549c4ef' GlobalEnqueue=(nil) ReleaseCount=0 
  PinInstance:  id='NBd42ae7cee549c4ef' GlobalEnqueue=(nil) 
  ReferenceList:  
    Reference:  Address=0x1f733f930 Handle=0x1f9587a00 Flags=DEP[01] 
      Timestamp=05-15-2018 07:00:05 InvalidatedFrom=0 
  LibraryObject:  Address=0x76c9e5fd0 HeapMask=0000-0011-0011-0000 Flags=EXS/LOC[0004] Flags2=/CSC[4000000] Flags3=[0000] PublicFlags=NST[0001] 
    Dependencies:  count='2' size='16' table='0x76c9e6e60' 
      Dependency:  num='0' 
        Reference=0x76c9e6288 Position=0 Flags=DEP[0001] 
        Handle=0x8b6a89c28 Type=SYNONYM(05) Parent=HFXF.EMP   
      Dependency:  num='1' 
        Reference=0x76c9e6320 Position=0 Flags=DEP[0001] 
        Handle=0x8b6d0e190 Type=PACKAGE(09) Parent=SYS.STANDARD   
    Accesses:  count='1' size='16' 
      Dependency:  num='0' Type=0011 
    DataBlocks:  
      Block:  #='0' name=KGLH0^48e92e5e pins=0 Change=NONE   
        Heap=0x39f836d98 Pointer=0x76c9e6110 Extent=0x76c9e5f10 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.468750 Size=3.937500 LoadTime=02-12-2025 02:28:06 
      Block:  #='4' name=PLMCD^48e92e5e pins=2 Change=NONE   
        Heap=0x76c9e63c8 Pointer=0xccd3a67f8 Extent=0xccd3a6738 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.195312 Size=3.937500 LoadTime=02-12-2025 02:28:06 KGI Error-3 = 9

LibraryHandle:  Address=0x39e333a08 Hash=48e92e5e LockMode=N PinMode=S LoadLockMode=0 Status=VALD Subpool=1 
  ObjectName:  Name=HFXF.PKG_XF_RS   
    FullHashValue=d42ae7cee549c4efb90f041348e92e5e Namespace=TABLE/PROCEDURE(01) Type=PACKAGE(09) ContainerId=0 ContainerUid=0 Identifier=2667149 OwnerI
dn=329 
  Statistics:  InvalidationCount=0 ExecutionCount=0 LoadCount=1 ActiveLocks=2 TotalLockCount=3 TotalPinCount=3 
  Counters:  BrokenCount=1 RevocablePointer=1 KeepDependency=0 Version=0 BucketInUse=0 HandleInUse=0 HandleReferenceCount=0 
  Concurrency:  DependencyMutex=0x39e333ab8(0, 15779417, 0, 0) Mutex=0x39e333b58(1275, 15779520, 92, 6) 
  Flags=PIN/TIM/[00002801] Flags2=[0000] 
  WaitersLists:  
    Lock=0x39e333a98[0x39e333a98,0x39e333a98] 
    Pin=0x39e333a78[0x39e333a78,0x39e333a78] 
    LoadLock=0x39e333af0[0x39e333af0,0x39e333af0] 
  Timestamp:  Current=05-15-2018 07:00:05 
  HandleReference:  Address=0x39e333c00 Handle=0x8afd36fa0 Flags=OWN[200] 
  LockInstance:  id='LBd42ae7cee549c4ef' GlobalEnqueue=(nil) ReleaseCount=0 
  PinInstance:  id='NBd42ae7cee549c4ef' GlobalEnqueue=(nil) 
  ReferenceList:  
    Reference:  Address=0x1f733f930 Handle=0x1f9587a00 Flags=DEP[01] 
      Timestamp=05-15-2018 07:00:05 InvalidatedFrom=0 
  LibraryObject:  Address=0x76c9e5fd0 HeapMask=0000-0011-0011-0000 Flags=EXS/LOC[0004] Flags2=/CSC[4000000] Flags3=[0000] PublicFlags=NST[0001] 
    Dependencies:  count='2' size='16' table='0x76c9e6e60' 
      Dependency:  num='0' 
        Reference=0x76c9e6288 Position=0 Flags=DEP[0001] 
        Handle=0x8b6a89c28 Type=SYNONYM(05) Parent=HFXF.EMP   
      Dependency:  num='1' 
        Reference=0x76c9e6320 Position=0 Flags=DEP[0001] 
        Handle=0x8b6d0e190 Type=PACKAGE(09) Parent=SYS.STANDARD   
    Accesses:  count='1' size='16' 
      Dependency:  num='0' Type=0011 
    DataBlocks:  
      Block:  #='0' name=KGLH0^48e92e5e pins=0 Change=NONE   
        Heap=0x39f836d98 Pointer=0x76c9e6110 Extent=0x76c9e5f10 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.468750 Size=3.937500 LoadTime=02-12-2025 02:28:06 
      Block:  #='4' name=PLMCD^48e92e5e pins=2 Change=NONE   
        Heap=0x76c9e63c8 Pointer=0xccd3a67f8 Extent=0xccd3a6738 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.195312 Size=3.937500 LoadTime=02-12-2025 02:28:06 KGI Error-3 = 9

LibraryHandle:  Address=0x39e333a08 Hash=48e92e5e LockMode=N PinMode=S LoadLockMode=0 Status=VALD Subpool=1 
  ObjectName:  Name=HFXF.PKG_XF_RS   
    FullHashValue=d42ae7cee549c4efb90f041348e92e5e Namespace=TABLE/PROCEDURE(01) Type=PACKAGE(09) ContainerId=0 ContainerUid=0 Identifier=2667149 OwnerI
dn=329 
  Statistics:  InvalidationCount=0 ExecutionCount=0 LoadCount=1 ActiveLocks=2 TotalLockCount=3 TotalPinCount=3 
  Counters:  BrokenCount=1 RevocablePointer=1 KeepDependency=0 Version=0 BucketInUse=0 HandleInUse=0 HandleReferenceCount=0 
  Concurrency:  DependencyMutex=0x39e333ab8(0, 15779419, 0, 0) Mutex=0x39e333b58(1275, 15779522, 92, 6) 
  Flags=PIN/TIM/[00002801] Flags2=[0000] 
  WaitersLists:  
    Lock=0x39e333a98[0x39e333a98,0x39e333a98] 
    Pin=0x39e333a78[0x39e333a78,0x39e333a78] 
    LoadLock=0x39e333af0[0x39e333af0,0x39e333af0] 
  Timestamp:  Current=05-15-2018 07:00:05 
  HandleReference:  Address=0x39e333c00 Handle=0x8afd36fa0 Flags=OWN[200] 
  LockInstance:  id='LBd42ae7cee549c4ef' GlobalEnqueue=(nil) ReleaseCount=0 
  PinInstance:  id='NBd42ae7cee549c4ef' GlobalEnqueue=(nil) 
  ReferenceList:  
    Reference:  Address=0x1f733f930 Handle=0x1f9587a00 Flags=DEP[01] 
      Timestamp=05-15-2018 07:00:05 InvalidatedFrom=0 
  LibraryObject:  Address=0x76c9e5fd0 HeapMask=0000-0011-0011-0000 Flags=EXS/LOC[0004] Flags2=/CSC[4000000] Flags3=[0000] PublicFlags=NST[0001] 
    Dependencies:  count='2' size='16' table='0x76c9e6e60' 
      Dependency:  num='0' 
        Reference=0x76c9e6288 Position=0 Flags=DEP[0001] 
        Handle=0x8b6a89c28 Type=SYNONYM(05) Parent=HFXF.EMP   
      Dependency:  num='1' 
        Reference=0x76c9e6320 Position=0 Flags=DEP[0001] 
        Handle=0x8b6d0e190 Type=PACKAGE(09) Parent=SYS.STANDARD   
    Accesses:  count='1' size='16' 
      Dependency:  num='0' Type=0011 
    DataBlocks:  
      Block:  #='0' name=KGLH0^48e92e5e pins=0 Change=NONE   
        Heap=0x39f836d98 Pointer=0x76c9e6110 Extent=0x76c9e5f10 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.468750 Size=3.937500 LoadTime=02-12-2025 02:28:06 
      Block:  #='4' name=PLMCD^48e92e5e pins=2 Change=NONE   
        Heap=0x76c9e63c8 Pointer=0xccd3a67f8 Extent=0xccd3a6738 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.195312 Size=3.937500 LoadTime=02-12-2025 02:28:06 KGI Error-3 = 9

LibraryHandle:  Address=0x39e333a08 Hash=48e92e5e LockMode=N PinMode=S LoadLockMode=0 Status=VALD Subpool=1 
  ObjectName:  Name=HFXF.PKG_XF_RS   
    FullHashValue=d42ae7cee549c4efb90f041348e92e5e Namespace=TABLE/PROCEDURE(01) Type=PACKAGE(09) ContainerId=0 ContainerUid=0 Identifier=2667149 OwnerI
dn=329 
  Statistics:  InvalidationCount=0 ExecutionCount=0 LoadCount=1 ActiveLocks=2 TotalLockCount=3 TotalPinCount=3 
  Counters:  BrokenCount=1 RevocablePointer=1 KeepDependency=0 Version=0 BucketInUse=0 HandleInUse=0 HandleReferenceCount=0 
  Concurrency:  DependencyMutex=0x39e333ab8(0, 15779421, 0, 0) Mutex=0x39e333b58(1275, 15779524, 92, 6) 
  Flags=PIN/TIM/[00002801] Flags2=[0000] 
  WaitersLists:  
    Lock=0x39e333a98[0x39e333a98,0x39e333a98] 
    Pin=0x39e333a78[0x39e333a78,0x39e333a78] 
    LoadLock=0x39e333af0[0x39e333af0,0x39e333af0] 
  Timestamp:  Current=05-15-2018 07:00:05 
  HandleReference:  Address=0x39e333c00 Handle=0x8afd36fa0 Flags=OWN[200] 
  LockInstance:  id='LBd42ae7cee549c4ef' GlobalEnqueue=(nil) ReleaseCount=0 
  PinInstance:  id='NBd42ae7cee549c4ef' GlobalEnqueue=(nil) 
  ReferenceList:  
    Reference:  Address=0x1f733f930 Handle=0x1f9587a00 Flags=DEP[01] 
      Timestamp=05-15-2018 07:00:05 InvalidatedFrom=0 
  LibraryObject:  Address=0x76c9e5fd0 HeapMask=0000-0011-0011-0000 Flags=EXS/LOC[0004] Flags2=/CSC[4000000] Flags3=[0000] PublicFlags=NST[0001] 
    Dependencies:  count='2' size='16' table='0x76c9e6e60' 
      Dependency:  num='0' 
        Reference=0x76c9e6288 Position=0 Flags=DEP[0001] 
        Handle=0x8b6a89c28 Type=SYNONYM(05) Parent=HFXF.EMP   
      Dependency:  num='1' 
        Reference=0x76c9e6320 Position=0 Flags=DEP[0001] 
        Handle=0x8b6d0e190 Type=PACKAGE(09) Parent=SYS.STANDARD   
    Accesses:  count='1' size='16' 
      Dependency:  num='0' Type=0011 
    DataBlocks:  
      Block:  #='0' name=KGLH0^48e92e5e pins=0 Change=NONE   
        Heap=0x39f836d98 Pointer=0x76c9e6110 Extent=0x76c9e5f10 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.468750 Size=3.937500 LoadTime=02-12-2025 02:28:06 
      Block:  #='4' name=PLMCD^48e92e5e pins=2 Change=NONE   
        Heap=0x76c9e63c8 Pointer=0xccd3a67f8 Extent=0xccd3a6738 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.195312 Size=3.937500 LoadTime=02-12-2025 02:28:06 KGI Error-3 = 9

LibraryHandle:  Address=0x39e333a08 Hash=48e92e5e LockMode=N PinMode=S LoadLockMode=0 Status=VALD Subpool=1 
  ObjectName:  Name=HFXF.PKG_XF_RS   
    FullHashValue=d42ae7cee549c4efb90f041348e92e5e Namespace=TABLE/PROCEDURE(01) Type=PACKAGE(09) ContainerId=0 ContainerUid=0 Identifier=2667149 OwnerI
dn=329 
  Statistics:  InvalidationCount=0 ExecutionCount=0 LoadCount=1 ActiveLocks=2 TotalLockCount=3 TotalPinCount=3 
  Counters:  BrokenCount=1 RevocablePointer=1 KeepDependency=0 Version=0 BucketInUse=0 HandleInUse=0 HandleReferenceCount=0 
  Concurrency:  DependencyMutex=0x39e333ab8(0, 15779424, 0, 0) Mutex=0x39e333b58(1275, 15779527, 92, 6) 
  Flags=PIN/TIM/[00002801] Flags2=[0000] 
  WaitersLists:  
    Lock=0x39e333a98[0x39e333a98,0x39e333a98] 
    Pin=0x39e333a78[0x39e333a78,0x39e333a78] 
    LoadLock=0x39e333af0[0x39e333af0,0x39e333af0] 
  Timestamp:  Current=05-15-2018 07:00:05 
  HandleReference:  Address=0x39e333c00 Handle=0x8afd36fa0 Flags=OWN[200] 
  LockInstance:  id='LBd42ae7cee549c4ef' GlobalEnqueue=(nil) ReleaseCount=0 
  PinInstance:  id='NBd42ae7cee549c4ef' GlobalEnqueue=(nil) 
  ReferenceList:  
    Reference:  Address=0x1f733f930 Handle=0x1f9587a00 Flags=DEP[01] 
      Timestamp=05-15-2018 07:00:05 InvalidatedFrom=0 
  LibraryObject:  Address=0x76c9e5fd0 HeapMask=0000-0011-0011-0000 Flags=EXS/LOC[0004] Flags2=/CSC[4000000] Flags3=[0000] PublicFlags=NST[0001] 
    Dependencies:  count='2' size='16' table='0x76c9e6e60' 
      Dependency:  num='0' 
        Reference=0x76c9e6288 Position=0 Flags=DEP[0001] 
        Handle=0x8b6a89c28 Type=SYNONYM(05) Parent=HFXF.EMP   
      Dependency:  num='1' 
        Reference=0x76c9e6320 Position=0 Flags=DEP[0001] 
        Handle=0x8b6d0e190 Type=PACKAGE(09) Parent=SYS.STANDARD   
    Accesses:  count='1' size='16' 
      Dependency:  num='0' Type=0011 
    DataBlocks:  
      Block:  #='0' name=KGLH0^48e92e5e pins=0 Change=NONE   
        Heap=0x39f836d98 Pointer=0x76c9e6110 Extent=0x76c9e5f10 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.468750 Size=3.937500 LoadTime=02-12-2025 02:28:06 
      Block:  #='4' name=PLMCD^48e92e5e pins=2 Change=NONE   
        Heap=0x76c9e63c8 Pointer=0xccd3a67f8 Extent=0xccd3a6738 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.195312 Size=3.937500 LoadTime=02-12-2025 02:28:06 KGI Error-3 = 9
Could not acquire mutex...Returning without dumping handle 0x39e333a08
KGI Error-3 = 9

LibraryHandle:  Address=0x39e333a08 Hash=48e92e5e LockMode=N PinMode=S LoadLockMode=0 Status=VALD Subpool=1 
  ObjectName:  Name=HFXF.PKG_XF_RS   
    FullHashValue=d42ae7cee549c4efb90f041348e92e5e Namespace=TABLE/PROCEDURE(01) Type=PACKAGE(09) ContainerId=0 ContainerUid=0 Identifier=2667149 OwnerI
dn=329 
  Statistics:  InvalidationCount=0 ExecutionCount=0 LoadCount=1 ActiveLocks=2 TotalLockCount=3 TotalPinCount=3 
  Counters:  BrokenCount=1 RevocablePointer=1 KeepDependency=0 Version=0 BucketInUse=0 HandleInUse=0 HandleReferenceCount=0 
  Concurrency:  DependencyMutex=0x39e333ab8(0, 15779427, 0, 0) Mutex=0x39e333b58(1275, 15779530, 92, 6) 
  Flags=PIN/TIM/[00002801] Flags2=[0000] 
  WaitersLists:  
    Lock=0x39e333a98[0x39e333a98,0x39e333a98] 
    Pin=0x39e333a78[0x39e333a78,0x39e333a78] 
    LoadLock=0x39e333af0[0x39e333af0,0x39e333af0] 
  Timestamp:  Current=05-15-2018 07:00:05 
  HandleReference:  Address=0x39e333c00 Handle=0x8afd36fa0 Flags=OWN[200] 
  LockInstance:  id='LBd42ae7cee549c4ef' GlobalEnqueue=(nil) ReleaseCount=0 
  PinInstance:  id='NBd42ae7cee549c4ef' GlobalEnqueue=(nil) 
  ReferenceList:  
    Reference:  Address=0x1f733f930 Handle=0x1f9587a00 Flags=DEP[01] 
      Timestamp=05-15-2018 07:00:05 InvalidatedFrom=0 
  LibraryObject:  Address=0x76c9e5fd0 HeapMask=0000-0011-0011-0000 Flags=EXS/LOC[0004] Flags2=/CSC[4000000] Flags3=[0000] PublicFlags=NST[0001] 
    Dependencies:  count='2' size='16' table='0x76c9e6e60' 
      Dependency:  num='0' 
        Reference=0x76c9e6288 Position=0 Flags=DEP[0001] 
        Handle=0x8b6a89c28 Type=SYNONYM(05) Parent=HFXF.EMP   
      Dependency:  num='1' 
        Reference=0x76c9e6320 Position=0 Flags=DEP[0001] 
        Handle=0x8b6d0e190 Type=PACKAGE(09) Parent=SYS.STANDARD   
    Accesses:  count='1' size='16' 
      Dependency:  num='0' Type=0011 
    DataBlocks:  
      Block:  #='0' name=KGLH0^48e92e5e pins=0 Change=NONE   
        Heap=0x39f836d98 Pointer=0x76c9e6110 Extent=0x76c9e5f10 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.468750 Size=3.937500 LoadTime=02-12-2025 02:28:06 
      Block:  #='4' name=PLMCD^48e92e5e pins=2 Change=NONE   
        Heap=0x76c9e63c8 Pointer=0xccd3a67f8 Extent=0xccd3a6738 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.195312 Size=3.937500 LoadTime=02-12-2025 02:28:06 KGI Error-3 = 9
Could not acquire mutex...Returning without dumping handle 0x39e333a08
KGI Error-3 = 9

LibraryHandle:  Address=0x39e333a08 Hash=48e92e5e LockMode=N PinMode=S LoadLockMode=0 Status=VALD Subpool=1 
  ObjectName:  Name=HFXF.PKG_XF_RS   
    FullHashValue=d42ae7cee549c4efb90f041348e92e5e Namespace=TABLE/PROCEDURE(01) Type=PACKAGE(09) ContainerId=0 ContainerUid=0 Identifier=2667149 OwnerI
dn=329 
  Statistics:  InvalidationCount=0 ExecutionCount=0 LoadCount=1 ActiveLocks=2 TotalLockCount=3 TotalPinCount=3 
  Counters:  BrokenCount=1 RevocablePointer=1 KeepDependency=0 Version=0 BucketInUse=0 HandleInUse=0 HandleReferenceCount=0 
  Concurrency:  DependencyMutex=0x39e333ab8(0, 15779431, 0, 0) Mutex=0x39e333b58(1275, 15779534, 92, 6) 
  Flags=PIN/TIM/[00002801] Flags2=[0000] 
  WaitersLists:  
    Lock=0x39e333a98[0x39e333a98,0x39e333a98] 
    Pin=0x39e333a78[0x39e333a78,0x39e333a78] 
    LoadLock=0x39e333af0[0x39e333af0,0x39e333af0] 
  Timestamp:  Current=05-15-2018 07:00:05 
  HandleReference:  Address=0x39e333c00 Handle=0x8afd36fa0 Flags=OWN[200] 
  LockInstance:  id='LBd42ae7cee549c4ef' GlobalEnqueue=(nil) ReleaseCount=0 
  PinInstance:  id='NBd42ae7cee549c4ef' GlobalEnqueue=(nil) 
  ReferenceList:  
    Reference:  Address=0x1f733f930 Handle=0x1f9587a00 Flags=DEP[01] 
      Timestamp=05-15-2018 07:00:05 InvalidatedFrom=0 
  LibraryObject:  Address=0x76c9e5fd0 HeapMask=0000-0011-0011-0000 Flags=EXS/LOC[0004] Flags2=/CSC[4000000] Flags3=[0000] PublicFlags=NST[0001] 
    Dependencies:  count='2' size='16' table='0x76c9e6e60' 
      Dependency:  num='0' 
        Reference=0x76c9e6288 Position=0 Flags=DEP[0001] 
        Handle=0x8b6a89c28 Type=SYNONYM(05) Parent=HFXF.EMP   
      Dependency:  num='1' 
        Reference=0x76c9e6320 Position=0 Flags=DEP[0001] 
        Handle=0x8b6d0e190 Type=PACKAGE(09) Parent=SYS.STANDARD   
    Accesses:  count='1' size='16' 
      Dependency:  num='0' Type=0011 
    DataBlocks:  
      Block:  #='0' name=KGLH0^48e92e5e pins=0 Change=NONE   
        Heap=0x39f836d98 Pointer=0x76c9e6110 Extent=0x76c9e5f10 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.468750 Size=3.937500 LoadTime=02-12-2025 02:28:06 
      Block:  #='4' name=PLMCD^48e92e5e pins=2 Change=NONE   
        Heap=0x76c9e63c8 Pointer=0xccd3a67f8 Extent=0xccd3a6738 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.195312 Size=3.937500 LoadTime=02-12-2025 02:28:06 KGI Error-3 = 9
Could not acquire mutex...Returning without dumping handle 0x39e333a08
KGI Error-3 = 9
Could not acquire mutex...Returning without dumping handle 0x39e333a08
KGI Error-3 = 9
Could not acquire mutex...Returning without dumping handle 0x39e333a08
KGI Error-3 = 9

LibraryHandle:  Address=0x39e333a08 Hash=48e92e5e LockMode=N PinMode=S LoadLockMode=0 Status=VALD Subpool=1 
  ObjectName:  Name=HFXF.PKG_XF_RS   
    FullHashValue=d42ae7cee549c4efb90f041348e92e5e Namespace=TABLE/PROCEDURE(01) Type=PACKAGE(09) ContainerId=0 ContainerUid=0 Identifier=2667149 OwnerI
dn=329 
  Statistics:  InvalidationCount=0 ExecutionCount=0 LoadCount=1 ActiveLocks=2 TotalLockCount=3 TotalPinCount=3 
  Counters:  BrokenCount=1 RevocablePointer=1 KeepDependency=0 Version=0 BucketInUse=0 HandleInUse=0 HandleReferenceCount=0 
  Concurrency:  DependencyMutex=0x39e333ab8(0, 15779436, 0, 0) Mutex=0x39e333b58(1275, 15779539, 92, 6) 
  Flags=PIN/TIM/[00002801] Flags2=[0000] 
  WaitersLists:  
    Lock=0x39e333a98[0x39e333a98,0x39e333a98] 
    Pin=0x39e333a78[0x39e333a78,0x39e333a78] 
    LoadLock=0x39e333af0[0x39e333af0,0x39e333af0] 
  Timestamp:  Current=05-15-2018 07:00:05 
  HandleReference:  Address=0x39e333c00 Handle=0x8afd36fa0 Flags=OWN[200] 
  LockInstance:  id='LBd42ae7cee549c4ef' GlobalEnqueue=(nil) ReleaseCount=0 
  PinInstance:  id='NBd42ae7cee549c4ef' GlobalEnqueue=(nil) 
  ReferenceList:  
    Reference:  Address=0x1f733f930 Handle=0x1f9587a00 Flags=DEP[01] 
      Timestamp=05-15-2018 07:00:05 InvalidatedFrom=0 
  LibraryObject:  Address=0x76c9e5fd0 HeapMask=0000-0011-0011-0000 Flags=EXS/LOC[0004] Flags2=/CSC[4000000] Flags3=[0000] PublicFlags=NST[0001] 
    Dependencies:  count='2' size='16' table='0x76c9e6e60' 
      Dependency:  num='0' 
        Reference=0x76c9e6288 Position=0 Flags=DEP[0001] 
        Handle=0x8b6a89c28 Type=SYNONYM(05) Parent=HFXF.EMP   
      Dependency:  num='1' 
        Reference=0x76c9e6320 Position=0 Flags=DEP[0001] 
        Handle=0x8b6d0e190 Type=PACKAGE(09) Parent=SYS.STANDARD   
    Accesses:  count='1' size='16' 
      Dependency:  num='0' Type=0011 
    DataBlocks:  
      Block:  #='0' name=KGLH0^48e92e5e pins=0 Change=NONE   
        Heap=0x39f836d98 Pointer=0x76c9e6110 Extent=0x76c9e5f10 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.468750 Size=3.937500 LoadTime=02-12-2025 02:28:06 
      Block:  #='4' name=PLMCD^48e92e5e pins=2 Change=NONE   
        Heap=0x76c9e63c8 Pointer=0xccd3a67f8 Extent=0xccd3a6738 Flags=I/-/P/A/-/-/- 
        FreedLocation=0 Alloc=1.195312 Size=3.937500 LoadTime=02-12-2025 02:28:06 KGI Error-3 = 9

LibraryHandle:  Address=0x39e333a08 Hash=48e92e5e LockMode=N PinMode=S LoadLockMode=0 Status=VALD Subpool=1 
  ObjectName:  Name=HFXF.PKG_XF_RS   
    FullHashValue=d42ae7cee549c4efb90f041348e92e5e Namespace=TABLE/PROCEDURE(01) Type=PACKAGE(09) ContainerId=0 ContainerUid=0 Identifier=2667149 OwnerI
dn=329 
  Statistics:  InvalidationCount=0 ExecutionCount=0 LoadCount=1 ActiveLocks=2 TotalLockCount=3 TotalPinCount=3 
  Counters:  BrokenCount=1 RevocablePointer=1 KeepDependency=0 Version=0 BucketInUse=0 HandleInUse=0 HandleReferenceCount=0 
  Concurrency:  DependencyMutex=0x39e333ab8(0, 15779438, 0, 0) Mutex=0x39e333b58(1275, 15779541, 92, 6) 
  Flags=PIN/TIM/[00002801] Flags2=[0000] 
  WaitersLists:  
    Lock=0x39e333a98[0x39e333a98,0x39e333a98] 
    Pin=0x39e333a78[0x39e333a78,0x39e333a78] 
    LoadLock=0x39e333af0[0x39e333af0,0x39e333af0] 
  Timestamp:  Current=05-15-2018 07:00:05 
  HandleReference:  Address=0x39e333c00 Handle=0x8afd36fa0 Flags=OWN[200] 
  LockInstance:  id='LBd42ae7cee549c4ef' GlobalEnqueue=(nil) ReleaseCount=0 
  PinInstance:  id='NBd42ae7cee549c4ef' GlobalEnqueue=(nil) 
  ReferenceList:  
    Reference:  Address=0x1f733f930 Handle=0x1f9587a00 Flags=DEP[01] 
      Timestamp=05-15-2018 07:00:05 InvalidatedFrom=0 
  LibraryObject:  Address=0x76c9e5fd0 HeapMask=0000-0011-0011-0000 Flags=EXS/LOC[0004] Flags2=/CSC[4000000] Flags3=[0000] PublicFlags=NST[0001] 
    Dependencies:  count='2' size='16' table='0x76c9e6e60' 
      Dependency:  num='0' 
        Reference=0x76c9e6288 Position=0 Flags=DEP[0001] 
        Handle=0x8b6a89c28 Type=SYNONYM(05) Parent=HFXF.EMP   
      Dependency:  num='1' 
        Reference=0x76c9e6320 Position=0 Flags=DEP[0001] 
        Handle=0x8b6d0e190 Type=PACKAGE(09) Parent=SYS.STANDARD   
    Accesses:  count='1' size='16' 
      Dependency:  num='0' Type=0011 
    DataBlocks:  

从trace中可以看到报错信息,KGI Error-3 = 9
Could not acquire mutex…Returning without dumping handle 0x39e333a08,handle 0x39e333a08是调用package HFXF.PKG_XF_RS 。
3、检查数据库对象

SQL>select object_name,owner,object_type,status from dba_objects where object_name='PKG_XF_RS';
OBJECT_NAME              OWNER  OBJECT_TYPE  STATUS
-------------------------------------------------------
PKG_XF_RS                HFXF	PACKAGE      VALID


SQL>select object_name,owner,object_type,status from dba_objects where object_name='F_GET_DATE';
no rows selected

HFXF用户下PACKAGE-PKG_XF_RS状态正常,PKG_XF_RS中调用的函数F_GET_DATE不存在。