Automated sundiag.sh for Exadata nodes

We recently had to collect sundiag.sh output to one of our Service Requests and I followed note “Oracle Exadata Diagnostic Information required for Disk Failures and some other Hardware issues (Doc ID 761868.1)“.

And while there isn’t too much of work involved I thought perhaps I could reduce few steps. Below is short script which creates folder to /tmp under running node (I use first database node), runs sundiag.sh via dcli on all nodes based on all_group file, downloads those files to running node and creates new archived file which you can upload to Service Request.

Before you run this you need to setup ssh equivalency across all the nodes.

Output when you run it looks like this:

[root@exa1dbadm01 diag]# sh get_sundiag_from_all_nodes.sh
Tue Jan 10 11:54:11 CET 2017: Getting sundiag scripts from all nodes
Tue Jan 10 11:54:11 CET 2017: /tmp/sundiag_output_20170110 exists
Tue Jan 10 11:54:11 CET 2017: Compressing old sundiag files and removing them exa1dbadm01: adding: var/log/exadatatmp/sundiag_exadbadm01_XXXXXXXXX_2017_01_10_11_46.tar.bz2 (deflated 1%)
exa1dbadm02: adding: var/log/exadatatmp/sundiag_exa1dbadm02_XXXXXXXXX_2017_01_10_11_46.tar.bz2 (deflated 1%)
exa1celadm01: adding: var/log/exadatatmp/sundiag_exa1celadm01_XXXXXXXXX_2017_01_10_11_46.tar.bz2 (deflated 2%)
exa1celadm02: adding: var/log/exadatatmp/sundiag_exa1celadm02_XXXXXXXXX_2017_01_10_11_46.tar.bz2 (deflated 1%)
exa1celadm03: adding: var/log/exadatatmp/sundiag_exa1celadm03_XXXXXXXXX_2017_01_10_11_46.tar.bz2 (deflated 1%)
Tue Jan 10 11:54:13 CET 2017: Running dcli sundiag.sh on all nodes
sundiag_output_20170110/sundiag_exa1celadm01_XXXXXXXXX_2017_01_10_11_54.tar.bz2
sundiag_output_20170110/sundiag_exa1celadm02_XXXXXXXXX_2017_01_10_11_54.tar.bz2
sundiag_output_20170110/sundiag_exa1celadm03_XXXXXXXXX_2017_01_10_11_54.tar.bz2
sundiag_output_20170110/sundiag_exa1dbadm01_XXXXXXXXX_2017_01_10_11_54.tar.bz2
sundiag_output_20170110/sundiag_exa1dbadm02_XXXXXXXXX_2017_01_10_11_54.tar.bz2
Tue Jan 10 11:59:04 CET 2017: Combined tar file of /tmp/sundiag_output_20170110 created as /tmp/exa_rack_sundiag_20170110_115839.tar.bz2
Tue Jan 10 11:59:04 CET 2017: You can review the logfile /tmp/sundiag_output_20170110/diagnostics_output_20170110.log for errors. All done now. Have a great day.

You can download the script from here.

Thoughts on this?

I have script under work which would run also ILOM snapshot through sundiag.sh if the ilom node is up but it needs to be done on different method as you need to supply root password during execution.