So you have a UNIX system admin emergency: A Solaris 10 workstation that was working and running is suddenly frozen and unresponsive.
Suspected culprit? Audit logs. If they have filled up and no other actions can be logged, all processes and attempted logins are stopped.
It is possible to get back into the frozen UNIX workstation and restore normal operation with a few steps:
Hit Stop-A to get to the Solaris OpenBoot PROM OK prompt. *
At the OpenBoot prompt, login to command mode with the firmware password if you have one set. (You should if you are worried about audit logs)
Type boot-s to boot into single user mode. You’ll need the firmware and root password.
Run a df –h to see what is mounted. If /var/audit is not mounted, the partition is probably full.
Find the entry in your /etc/vfstab file for the disk device that contains /var/audit.
Mount the disk device (not the raw rdsk device) using the mount command with your disk device name such as:
mount /dev/dsk/c1t0d0s7 /var/audit
Run a df –h again just for a sanity check. /var/audit should show as mounted and at 100% capacity.
Change directory to /var/audit and run gzip on the largest audit files or move them according to your local security policy.
After gzip compression completes, (this may take a while if the files are large) run df –h again to check that /var/audit is under 80% capacity or so.
A test compression on the binary audit files using gzip on my system dropped the /var/audit partition from 100% to 5% capacity. Your results may vary.
While still in single user mode, type init 6 to reboot and go back to normal multi-user mode. All your normal drive partitions and network devices should be mounted and available.
*This works on Solaris SPARC machines, x86 machines have no built-in Open Boot PROM so you must use GRUB or other boot loader to boot to single user mode.