ORA-09925: Unable to create audit trail file Linux-x86_64 Error: 28: No space left on devic

晚上,接到信息科值班电话,说一个生产库连不进去了。

于是v*n上去,用工具连也连不进去,报错:

ORA-09925: Unable to create audit trail file
Linux-x86_64 Error: 28: No space left on device
Additional information: 9925
ORA-02002: error while writing to audit trail
ORA-09925: Unable to create audit trail file
Linux-x86_64 Error: 28: No space left on device
Additional information: 9925
09925. 00000 -   "Unable to create audit trail file"
*Cause:    ORACLE was not able to create the file being used to hold
           audit trail records.
*Action:   Check the UNIX error number for a possible operating system error.
           If there is no error, contact ORACLE customer support.
供应商代码 9925

看到No space于是df -h发现磁盘未满呀。

sqlplus 也登不进去:

ERROR:
ORA-09925: Unable to create audit trail file
Linux-x86_64 Error: 28: No space left on device
Additional information: 9925
ORA-01075: you are currently logged on

打开日志文件查看:

ORA-09925: Unable to create audit trail file Linux-x86_64 Error: 28: No space left on devic

刚发生的情况,可能和新上系统有关系。

Audit file写不进去,Audit file在哪里呢?找到路径发现目录下文件很多,显示不完。于是直接 rm -rf,报错:

-bash: /bin/rm: Argument list too long

网上搜索答案:https://www.cnblogs.com/tzhyy/p/10712339.html

Oracle11g <wbr>adump目录下面.aud增长导致空间撑满无法删除导致CRS无法启动的解决方法

输入下面的命令,删除成功:

ls | xargs -n 10 rm -fr ls

命令解释为:输出所有的文件名(用空格分割) xargs就是将ls的输出,每10个为一组(以空格为分隔符),

作为rm -rf的参数也就是说将所有文件名10个为一组,由rm -rf删除

分析原因

发生上面错误的原因 有以下三种可能:

① 磁盘空间已满,导致无法创建审核文件,使用df -h查询磁盘空间;

② 磁盘inodes已经占用完了,导致无法打开新的文件,使用df -i查看inode使用情况;

③ 路径权限不足,无法创建审核文件,查看路径权限情况ls -l。

现在确定是 磁盘inodes已经占用完引起的。

# df -i  可查看inodes使用情况
Filesystem            Inodes  IUsed   IFree  IUse%  Mounted  on
/dev/mapper/rootvg-root 2441216  2943  2438273    1%  / 
......
/dev/mapper/dbvg-oraclelv 51432   50950   482    100%  /oracle

查看SUD文件:

[[email protected] adump]# more orcl2_ora_17679_20200306210542452850143795.aud 
Audit file /u01/app/oracle/admin/ORCL/adump/xxxxxx2_ora_17679_20200306210542452850143795.aud
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1
System name:    Linux
Node name:    ORCL
Release:    2.6.32-431.el6.x86_64
Version:    #1 SMP Sun Nov 10 22:19:54 EST 2013
Machine:    x86_64
Instance name: ORCL2
Redo thread mounted by this instance: 1
Oracle process number: 72
Unix process pid: 17679, image: [email protected]

Fri Mar  6 21:05:42 2020 +08:00
LENGTH: "357"
SESSIONID:[10] "4294967295" ENTRYID:[1] "1" STATEMENT:[1] "1" USERID:[5] "USER" USERHOST:[10] "server2897" TERMINAL:[7] "
unknown" ACTION:[3] "100" RETURNCODE:[1] "0" COMMENT$TEXT:[98] "Authenticated by: DATABASE; Client address: (ADDRESS=(PROT
OCOL=tcp)(HOST=10.10.28.97)(PORT=62616))" OS$USERID:[13] "Administrator" DBID:[9] "509355099" PRIV$USED:[1] "5"

Fri Mar  6 21:05:42 2020 +08:00
LENGTH: "229"
SESSIONID:[10] "4294967295" ENTRYID:[1] "1" USERID:[5] "U1228" ACTION:[3] "101" RETURNCODE:[1] "0" LOGOFF$PREAD:[1] "0" LO
GOFF$LREAD:[5] "10302" LOGOFF$LWRITE:[1] "0" LOGOFF$DEAD:[1] "0" DBID:[9] "509355099" SESSIONCPU:[2] "10"

结论:由上面服务器某个用户引起,联系程序提供商解决。

暂时解决办法:

(1)删除:ls | xargs -n 10 rm -fr ls

(2)配置crontab -e 定时删除:15 * * * * find /u01/app/oracle/admin/ORCL/adump/ -name "*aud" -exec rm -rf {} \;

长期解决办法:

(1)联系应用服务商查找原因;

(2)配置sqlnet.ora文件增加(未确定是否起作用) :SQLNET.AUTHENTICATION_SERVICES = (NONE)