“Space is running low in /ddvar. Please clear space or core dump capability will be disabled.”
Today I stumbled upon an interesting issue while prepping for a Data Domain upgrade. The first step taken to prepare for a Data Domain OS upgrade is to reboot the appliance beforehand. This ensures no issues are present before upgrading. If issues are present, they will most likely be brought to light before the upgrade. This is especially important if your system has been up for an extended period of time!
So you can imagine I was surprised to see the following message upon successful reboot of my appliance.
This error message indicates that the /ddvar directory is getting close to be full. This directory is important since it stores core dumps of the appliance, a critical piece of information needed by support when determining the cause of file system panics and/or other system issues.
Other variations of the alerts are as follows:
- Alert “Space running low in /ddvar for system maintenance activities.”
- Alert: “Space is running low in /ddvar. Please clear space or core dump capability will be disabled.”
- Alert: “Core dump capability is now disabled due to lack of space in /ddvar”
To see how much space was left, I headed over to Data Management menu within the GUI of the appliance.
.
As you can see, the summary tab indicates that the /ddvar directory has increased utilization (only 14.1 GB available, 60.7 used).
In order to clean up space within the /ddvar directory, I headed over to PuTTY and logged into the appliance. I ran the “support coredump list” command to obtain a list of files.
**********************************************************
login as: stephen
Data Domain OS
Using keyboard-interactive authentication.
Password:
Last login: Tue Mar 20 09:43:50 CDT 2018
Welcome to Data Domain OS 5.7.1.0-522982
—————————————-
**
** NOTICE: There is 1 outstanding alert. Run “alerts show current”
** to display outstanding alert(s).
**
stephen# support coredump list
File Size Time Created
—————————– ———- ————————
ddfs.core.11762.1398126501.gz 2931.0 MiB Mon Apr 21 19:45:30 2014
ddfs.core.12820.1481026154.gz 5604.4 MiB Tue Dec 6 06:25:31 2016
ddfs.core.16057.1499353534.gz 7060.0 MiB Thu Jul 6 10:24:29 2017
ddfs.core.21871.1520982774.gz 6605.8 MiB Tue Mar 13 18:47:49 2018
ddfs.core.27399.1519887699.gz 8356.3 MiB Thu Mar 1 01:40:20 2018
ddfs.core.8206.1404200742.gz 5094.3 MiB Tue Jul 1 03:02:57 2014
ddfs.core.8716.1482235743.gz 4185.0 MiB Tue Dec 20 06:22:07 2016
ddfs.core.887.1398202130.gz 2803.4 MiB Tue Apr 22 17:09:02 2014
ddfs.core.954.1403257282.gz 4616.2 MiB Fri Jun 20 04:55:20 2014
sms.core.1763.1425911004.gz 5.9 MiB Mon Mar 9 09:23:25 2015
sms.core.26739.1382479353.gz 5.1 MiB Tue Oct 22 17:02:35 2013
sms.core.31832.1400614763.gz 5.8 MiB Tue May 20 14:39:24 2014
sms.core.9648.1370800834.gz 5.2 MiB Sun Jun 9 13:00:35 2013
snmpd.core.2790.1459368292.gz 26.5 MiB Wed Mar 30 15:04:53 2016
—————————– ———- ————————
**********************************************************
From the above text, you can see there are several historical core dumps that can be deleted. Once these core dump files are deleted, the /ddvar directory will have sufficient space for future core dumps.
To delete a coredump file, simply type the following command
support coredump delete ddfs.core.27399.1519887699.gz
The file has been successfully deleted. I now have all alerts cleared, and sufficient space within my /ddvar directory.
For more information on this topic, please visit Dell EMC’s website here.
**IMPORTANT NOTE** If you have many of these errors, this indicates that you are having file system panics and/or other issues. It is very important to open a case with support to determine why your Data Domain is having these issues. File system panics are not normal, and should be investigated in order to provide a root cause analysis. While this is a feature that most users can perform, always open a support SR if you are unsure about deleting anything on your appliance. Dell EMC support should always be the final say. It’s always better to error on the side of caution. Finally, it’s best to keep the last few coredumps available on your system. If you delete them, support will not be able to look back and have a history/snapshot of your system or its behavior. Use at your own risk.
TheVirtualBoi 