3 Replies Latest reply on Jan 30, 2015 12:35 AM by swatsk

    Apple OSX/net-snmpd

    joeytmann

      Have a bit of an interesting issue with some Apple servers we are trying to monitor. All systems are running the default Apple installed net-snmpd 5.4.2.1. How ever on a handful of machines if we poll the CPU the snmpd process crashes and restarts(shows in Orion as a reboot of the system). The snmpd.conf file is identical across all servers. In comparing the hardware of two of the servers, one that has the problem and one that doesn't, but have the exact same OS and hardware. Here is what we get in the system log.

      09:42:38.259 ReportCrash[50717:2503] Saved crash report for snmpd[49867] version ??? (???) to /Library/Logs/DiagnosticReports/snmpd_2011-10-03-094238_localhost.crash

       

      Then looking at this crash file we see the following. I have been googling around but not finding much info. Anyone have any ideas. For now I have turned off CPU monitoring on these systems, about a half a dozen. Interesting thing, I have APM doing a process monitor on the AFP service and get CPU info from it.

       

      Process:         snmpd [49867]

      Path:            /usr/sbin/snmpd

      Identifier:      snmpd

      Version:         ??? (???)

      Code Type:       X86-64 (Native)

      Parent Process:  launchd [1]

       

      Date/Time:       2011-10-03 09:42:38.007 -0500

      OS Version:      Mac OS X Server 10.6.8 (10K549)

      Report Version:  6

       

      Exception Type:  EXC_ARITHMETIC (SIGFPE) Exception Codes: EXC_I386_DIV (divide by zero) Crashed Thread:  0  Dispatch queue: com.apple.main-thread

       

      Thread 0 Crashed:  Dispatch queue: com.apple.main-thread

      0   libnetsnmpmibs.15.dylib         0x000000010006b437 var_hrproc + 169

      1   libnetsnmphelpers.15.dylib                   0x000000010004805c

      netsnmp_old_api_helper + 390

      2   libnetsnmpagent.15.dylib      0x0000000100016f95

      netsnmp_call_handlers + 868

      3   libnetsnmpagent.15.dylib      0x0000000100009a83 handle_var_requests

      + 91

      4   libnetsnmpagent.15.dylib      0x000000010000c179 handle_pdu + 524

      5   libnetsnmpagent.15.dylib      0x000000010000d0af

      netsnmp_handle_request + 500

      6   libnetsnmpagent.15.dylib      0x000000010000d443 handle_snmp_packet +

      445

      7   libnetsnmp.15.dylib                   0x000000010015462f snmp_timeout + 3928

      8   libnetsnmp.15.dylib                   0x0000000100155605 _sess_read + 3719

      9   libnetsnmp.15.dylib                   0x00000001001556e5 snmp_read + 30

      10  snmpd                           0x000000010000284e 0x100000000 + 10318

      11  snmpd                           0x0000000100000d40 0x100000000 + 3392

       

      Thread 1:  Dispatch queue: com.apple.libdispatch-manager

      0   libSystem.B.dylib                        0x00007fff89f44c0a kevent + 10

      1   libSystem.B.dylib                        0x00007fff89f46add _dispatch_mgr_invoke

      + 154

      2   libSystem.B.dylib                        0x00007fff89f467b4

      _dispatch_queue_invoke + 185

      3   libSystem.B.dylib                        0x00007fff89f462de

      _dispatch_worker_thread2 + 252

      4   libSystem.B.dylib                        0x00007fff89f45c08 _pthread_wqthread +

      353

      5   libSystem.B.dylib                        0x00007fff89f45aa5 start_wqthread + 13

       

      Thread 0 crashed with X86 Thread State (64-bit):

        rax: 0x0000000000000000  rbx: 0x00007fff5fbff8b0  rcx:

      0x000000010101ac00  rdx: 0x0000000000000000

        rdi: 0x0000000000000000  rsi: 0x0000000100035f40  rbp:

      0x00007fff5fbfec10  rsp: 0x00007fff5fbfec00

         r8: 0x00007fff5fbff8b0   r9: 0x00007fff5fbff8b8  r10:

      0x0000000000000001  r11: 0x00007fff5fbff0a0

        r12: 0x00007fff5fbfec80  r13: 0x0000000100501810  r14:

      0x000000000000000c  r15: 0x0000000000000060

        rip: 0x000000010006b437  rfl: 0x0000000000010246  cr2: 0x00000000000e1000

       

      Binary Images:

             0x100000000 -        0x100003ff7 +snmpd ??? (???)

      <CA42A2E4-6BDE-85AC-FAAE-6FCA987193C4> /usr/sbin/snmpd

             0x100008000 -        0x100034ff7  libnetsnmpagent.15.dylib 17.2.0

      (compatibility 17.0.0) <98039749-650B-6F76-6325-737B36BCD895>

      /usr/lib/libnetsnmpagent.15.dylib

             0x100042000 -        0x10005cfff  libnetsnmphelpers.15.dylib 17.2.0

      (compatibility 17.0.0) <737E6394-3B05-3FED-357B-B4FF07A8658B>

      /usr/lib/libnetsnmphelpers.15.dylib

             0x100067000 -        0x1000f0fff  libnetsnmpmibs.15.dylib 17.2.0

      (compatibility 17.0.0) <7F7CD239-BE1A-0199-D2A7-2E85CF4679B5>

      /usr/lib/libnetsnmpmibs.15.dylib

             0x10012f000 -        0x1001a9ff7  libnetsnmp.15.dylib 17.2.0

      (compatibility 17.0.0) <2BE5579E-2ADF-2930-FA8E-02E9CBB101E3>

      /usr/lib/libnetsnmp.15.dylib

          0x7fff5fc00000 -     0x7fff5fc3bdef  dyld 132.1 (???)

      <B536F2F1-9DF1-3B6C-1C2C-9075EA219A06> /usr/lib/dyld

          0x7fff80139000 -     0x7fff80160ff7  libJPEG.dylib ??? (???)

      <46A413EA-4FD1-A050-2EF0-6279F3EAD581>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/ImageIO.framework/Versions/A/Resources/libJPEG.dylib

          0x7fff801a2000 -     0x7fff804d6fef  com.apple.CoreServices.CarbonCore

      861.39 (861.39) <1386A24D-DD15-5903-057E-4A224FAF580B>

      /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Car

      bonCore.framework/Versions/A/CarbonCore

          0x7fff804d7000 -     0x7fff80571fe7  com.apple.ApplicationServices.ATS

      275.16 (???) <4B70A2FC-1902-5F27-5C3B-5C78C283C6EA>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/ATS.framework/Versions/A/ATS

          0x7fff80572000 -     0x7fff805acfff  libcups.2.dylib 2.8.0

      (compatibility 2.0.0) <7982734A-B66B-44AA-DEEC-364D2C10009B>

      /usr/lib/libcups.2.dylib

          0x7fff8072a000 -     0x7fff8076bfef  com.apple.QD 3.36 (???)

      <5DC41E81-32C9-65B2-5528-B33E934D5BB4>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/QD.framework/Versions/A/QD

          0x7fff8094e000 -     0x7fff809aefe7  com.apple.framework.IOKit 2.0

      (???) <4F071EF0-8260-01E9-C641-830E582FA416>

      /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit

          0x7fff809d3000 -     0x7fff80b91ff7  com.apple.ImageIO.framework 3.0.4

      (3.0.4) <0A4F51A1-4502-767B-8A4E-F14C6214EF88>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/ImageIO.framework/Versions/A/ImageIO

          0x7fff80d7b000 -     0x7fff80dc3ff7  libvDSP.dylib 268.0.1

      (compatibility 1.0.0) <98FC4457-F405-0262-00F7-56119CA107B6>

      /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLi

      b.framework/Versions/A/libvDSP.dylib

          0x7fff80dee000 -     0x7fff81070fe7  com.apple.Foundation 6.6.7

      (751.62) <6F2A5BBF-6990-D561-2928-AD61E94036D9>

      /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation

          0x7fff810e7000 -     0x7fff81370ff7  com.apple.security 6.1.2 (55002)

      <4419AFFC-DAE7-873E-6A7D-5C9A5A4497A6>

      /System/Library/Frameworks/Security.framework/Versions/A/Security

          0x7fff81401000 -     0x7fff81c0bfe7  libBLAS.dylib 219.0.0

      (compatibility 1.0.0) <FC941ECB-71D0-FAE3-DCBF-C5A619E594B8>

      /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLi

      b.framework/Versions/A/libBLAS.dylib

          0x7fff81c0e000 -     0x7fff81d2dfe7  libcrypto.0.9.8.dylib 0.9.8

      (compatibility 0.9.8) <14115D29-432B-CF02-6B24-A60CC533A09E>

      /usr/lib/libcrypto.0.9.8.dylib

          0x7fff81f60000 -     0x7fff8203dfff  com.apple.vImage 4.1 (4.1)

      <C3F44AA9-6F71-0684-2686-D3BBC903F020>

      /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vImag

      e.framework/Versions/A/vImage

          0x7fff825f6000 -     0x7fff8261efff  com.apple.DictionaryServices

      1.1.2 (1.1.2) <E9269069-93FA-2B71-F9BA-FDDD23C4A65E>

      /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Dic

      tionaryServices.framework/Versions/A/DictionaryServices

          0x7fff8261f000 -     0x7fff82623ff7  libmathCommon.A.dylib 315.0.0

      (compatibility 1.0.0) <95718673-FEEE-B6ED-B127-BCDBDB60D4E5>

      /usr/lib/system/libmathCommon.A.dylib

          0x7fff82ad2000 -     0x7fff82af3fff  libresolv.9.dylib 41.0.0

      (compatibility 1.0.0) <6993F348-428F-C97E-7A84-7BD2EDC46A62>

      /usr/lib/libresolv.9.dylib

          0x7fff82afb000 -     0x7fff82b78fef  libstdc++.6.dylib 7.9.0

      (compatibility 7.0.0) <35ECA411-2C08-FD7D-11B1-1B7A04921A5C>

      /usr/lib/libstdc++.6.dylib

          0x7fff82eeb000 -     0x7fff82fa1ff7  libobjc.A.dylib 227.0.0

      (compatibility 1.0.0) <03140531-3B2D-1EBA-DA7F-E12CC8F63969>

      /usr/lib/libobjc.A.dylib

          0x7fff82fa2000 -     0x7fff8369eff7  com.apple.CoreGraphics 1.545.0

      (???) <58D597B1-EB3B-710E-0B8C-EC114D54E11B>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/CoreGraphics.framework/Versions/A/CoreGraphics

          0x7fff836e4000 -     0x7fff836f5ff7  libz.1.dylib 1.2.3 (compatibility

      1.0.0) <FB5EE53A-0534-0FFA-B2ED-486609433717> /usr/lib/libz.1.dylib

          0x7fff83848000 -     0x7fff83857fff  com.apple.NetFS 3.2.2 (3.2.2)

      <7CCBD70E-BF31-A7A7-DB98-230687773145>

      /System/Library/Frameworks/NetFS.framework/Versions/A/NetFS

          0x7fff83983000 -     0x7fff83983ff7  com.apple.Accelerate 1.6

      (Accelerate 1.6) <15DF8B4A-96B2-CB4E-368D-DEC7DF6B62BB>

      /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate

          0x7fff83984000 -     0x7fff83a14fff  com.apple.SearchKit 1.3.0 (1.3.0)

      <4175DC31-1506-228A-08FD-C704AC9DF642>

      /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Sea

      rchKit.framework/Versions/A/SearchKit

          0x7fff83a15000 -     0x7fff83acefff  libsqlite3.dylib 9.6.0

      (compatibility 9.0.0) <2C5ED312-E646-9ADE-73A9-6199A2A43150>

      /usr/lib/libsqlite3.dylib

          0x7fff83af9000 -     0x7fff83af9ff7  com.apple.CoreServices 44 (44)

      <DC7400FB-851E-7B8A-5BF6-6F50094302FB>

      /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices

          0x7fff842f1000 -     0x7fff843b2fef  com.apple.ColorSync 4.6.6 (4.6.6)

      <BB2C5813-C61D-3CBA-A8F7-0E59E46EBEE8>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/ColorSync.framework/Versions/A/ColorSync

          0x7fff84417000 -     0x7fff844d8fff  libFontParser.dylib ??? (???)

      <A00BB0A7-E46C-1D07-1391-194745566C7E>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/ATS.framework/Versions/A/Resources/libFontParser.dylib

          0x7fff845bf000 -     0x7fff8477dfff  libicucore.A.dylib 40.0.0

      (compatibility 1.0.0) <4274FC73-A257-3A56-4293-5968F3428854>

      /usr/lib/libicucore.A.dylib

          0x7fff84b4b000 -     0x7fff84b4cff7  com.apple.TrustEvaluationAgent

      1.1 (1) <74800EE8-C14C-18C9-C208-20BBDB982D40>

      /System/Library/PrivateFrameworks/TrustEvaluationAgent.framework/Versions/A

      /TrustEvaluationAgent

          0x7fff84b4d000 -     0x7fff84b8efff  com.apple.SystemConfiguration

      1.10.8 (1.10.2) <78D48D27-A9C4-62CA-2803-D0BBED82855A>

      /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemC

      onfiguration

          0x7fff84c88000 -     0x7fff84d9ffef  libxml2.2.dylib 10.3.0

      (compatibility 10.0.0) <1B27AFDD-DF87-2009-170E-C129E1572E8B>

      /usr/lib/libxml2.2.dylib

          0x7fff85f02000 -     0x7fff85f55ff7  com.apple.HIServices 1.8.3 (???)

      <F6E0C7A7-C11D-0096-4DDA-2C77793AA6CD>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/HIServices.framework/Versions/A/HIServices

          0x7fff86be5000 -     0x7fff86be7fff  libRadiance.dylib ??? (???)

      <A9DB4D5D-4072-971B-DEF6-DDE645F415EA>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/ImageIO.framework/Versions/A/Resources/libRadiance.dylib

          0x7fff86be8000 -     0x7fff86bfdff7  com.apple.LangAnalysis 1.6.6

      (1.6.6) <DC999B32-BF41-94C8-0583-27D9AB463E8B>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/LangAnalysis.framework/Versions/A/LangAnalysis

          0x7fff86c7a000 -     0x7fff86c7aff7  com.apple.Accelerate.vecLib 3.6

      (vecLib 3.6) <4CCE5D69-F1B3-8FD3-1483-E0271DB2CCF3>

      /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLi

      b.framework/Versions/A/vecLib

          0x7fff86c7b000 -     0x7fff86d2bfff  edu.mit.Kerberos 6.5.11 (6.5.11)

      <085D80F5-C9DC-E252-C21B-03295E660C91>

      /System/Library/Frameworks/Kerberos.framework/Versions/A/Kerberos

          0x7fff86d2c000 -     0x7fff86d7bfef  libTIFF.dylib ??? (???)

      <1E2593D1-A7F6-84C6-DF8F-0B46AE445926>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/ImageIO.framework/Versions/A/Resources/libTIFF.dylib

          0x7fff86dff000 -     0x7fff86e1cff7  libPng.dylib ??? (???)

      <6D8E515B-E0A2-2BA1-9CAC-8CB8A8B35879>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/ImageIO.framework/Versions/A/Resources/libPng.dylib

          0x7fff87299000 -     0x7fff872c4ff7  libxslt.1.dylib 3.24.0

      (compatibility 3.0.0) <8AB4CA9E-435A-33DA-7041-904BA7FA11D5>

      /usr/lib/libxslt.1.dylib

          0x7fff872c5000 -     0x7fff87365fff  com.apple.LaunchServices 362.3

      (362.3) <B90B7C31-FEF8-3C26-BFB3-D8A48BD2C0DA>

      /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Lau

      nchServices.framework/Versions/A/LaunchServices

          0x7fff87597000 -     0x7fff875d2fff  com.apple.AE 496.5 (496.5)

      <208DF391-4DE6-81ED-C697-14A2930D1BC6>

      /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/AE.

      framework/Versions/A/AE

          0x7fff875d3000 -     0x7fff875d9ff7  com.apple.DiskArbitration 2.3

      (2.3) <857F6E43-1EF4-7D53-351B-10DE0A8F992A>

      /System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitra

      tion

          0x7fff8795f000 -     0x7fff879a9ff7  com.apple.Metadata 10.6.3

      (507.15) <2EF19055-D7AE-4D77-E589-7B71B0BC1E59>

      /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Met

      adata.framework/Versions/A/Metadata

          0x7fff87a80000 -     0x7fff87a96fef  libbsm.0.dylib ??? (???)

      <42D3023A-A1F7-4121-6417-FCC6B51B3E90> /usr/lib/libbsm.0.dylib

          0x7fff87a97000 -     0x7fff87b1cff7

      com.apple.print.framework.PrintCore 6.3 (312.7) <CDFE82DD-D811-A091-179F-6E76069B432D>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/PrintCore.framework/Versions/A/PrintCore

          0x7fff87b1d000 -     0x7fff87b87fe7  libvMisc.dylib 268.0.1

      (compatibility 1.0.0) <514D400C-50A5-C196-83AA-1035DDC8FBBE>

      /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLi

      b.framework/Versions/A/libvMisc.dylib

          0x7fff87c82000 -     0x7fff87c82ff7  com.apple.ApplicationServices 38

      (38) <10A0B9E9-4988-03D4-FC56-DDE231A02C63>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Applica

      tionServices

          0x7fff87c97000 -     0x7fff87c9cfff  libGIF.dylib ??? (???)

      <201B8077-B5CC-11AA-E1B0-1D057ABE416A>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/ImageIO.framework/Versions/A/Resources/libGIF.dylib

          0x7fff89023000 -     0x7fff89031ff7  libkxld.dylib ??? (???)

      <8145A534-95CC-9F3C-B78B-AC9898F38C6F> /usr/lib/system/libkxld.dylib

          0x7fff89032000 -     0x7fff891a9fe7  com.apple.CoreFoundation 6.6.5

      (550.43) <31A1C118-AD96-0A11-8BDF-BD55B9940EDC>

      /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundati

      on

          0x7fff892a3000 -     0x7fff892b7ff7

      com.apple.speech.synthesis.framework 3.10.35 (3.10.35) <621B7415-A0B9-07A7-F313-36BEEDD7B132>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/SpeechSynthesis.framework/Versions/A/SpeechSynthesis

          0x7fff89378000 -     0x7fff897bbfef  libLAPACK.dylib 219.0.0

      (compatibility 1.0.0) <0CC61C98-FF51-67B3-F3D8-C5E430C201A9>

      /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLi

      b.framework/Versions/A/libLAPACK.dylib

          0x7fff89ceb000 -     0x7fff89dbffe7  com.apple.CFNetwork 454.12.4

      (454.12.4) <C83E2BA1-1818-B3E8-5334-860AD21D1C80>

      /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CFN

      etwork.framework/Versions/A/CFNetwork

          0x7fff89f2b000 -     0x7fff8a0ecfef  libSystem.B.dylib 125.2.11

      (compatibility 1.0.0) <9AB4F1D1-89DC-0E8A-DC8E-A4FE4D69DB69>

      /usr/lib/libSystem.B.dylib

          0x7fff8a0ed000 -     0x7fff8a1aafff  com.apple.CoreServices.OSServices

      359.2 (359.2) <BBB8888E-18DE-5D09-3C3A-F4C029EC7886>

      /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/OSS

      ervices.framework/Versions/A/OSServices

          0x7fff8a4f0000 -     0x7fff8a53cfff  libauto.dylib ??? (???)

      <F7221B46-DC4F-3153-CE61-7F52C8C293CF> /usr/lib/libauto.dylib

          0x7fff8a5c7000 -     0x7fff8a645ff7  com.apple.CoreText 151.10 (???)

      <54961997-55D8-DC0F-2634-674E452D5A8E>

      /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Framewo

      rks/CoreText.framework/Versions/A/CoreText

          0x7fffffe00000 -     0x7fffffe01fff  libSystem.B.dylib ??? (???)

      <9AB4F1D1-89DC-0E8A-DC8E-A4FE4D69DB69> /usr/lib/libSystem.B.dylib