9 Replies Latest reply on Nov 6, 2008 10:08 AM by Karlo.Zatylny

    Avg Min/Max Charts exceeding port Speed

      I have Cisco 6500 series switches with GigE interface blades.  Some of the interfaces are capped at 10 Mbps.  The Avg Min/Max/Average bps In/Out Charts will sometimes show Max IN and/or Max OUT spikes that exceed 10Mbps.  How is it possible that traffic is exceeding the forced 10Mbps?  I’ve pulled raw traffic metrics from InterfaceTraffic_Details table and the chart is plotting what is in the DB.  Has anyone else seen this?

      Interface polling clarification for troubleshooting.  I’m not exactly sure what data is from the switch and what is being calculated and filled in by Orion.  I was assuming ifINOctects and IfOutOctects where being retrieved out of SNMP ifTable and entered into the Interfaces DB table at inbps and outbps during poll time.  Then at the stat rate, put into the InterfaceTraffic_ Details for Min/MaxIN and Min/MaxOUT and In/Out_Avergagebps where derived from there.   Is this correct?

       

      Running
      Orion 8.5.1 w/SP3

       

      Thanks,

      Joe 

        • Re: Avg Min/Max Charts exceeding port Speed
          aLTeReGo

          What method are you using to "cap" the bandwidth at 10Mbps?

            • Re: Avg Min/Max Charts exceeding port Speed

              We force some of the GigE interfaces to speed 10, duplex FULL on the switches.  Those are the interfaces that I notice spikes plotted greater than 10Mbit.  Orion also indicates the port speed is only 10Mbit, "10000000".  I would expect the graphs to show traffic less than 10Mbps.  Perhaps there is Max/MIN IN/OUT AVG math being done is calculating the values greater than 10Mbit and then plotting.  I'm not 100% sure which values in the InterfacesTraffic_Details tables are originating (raw values) from the switch and which are just calculated by Orion after receiving raw data from the switch to tell whether it is in fact the math.  Curious, if anyone else has experienced traffic plotted at greater speeds than interface provides.

                • Re: Avg Min/Max Charts exceeding port Speed
                  aLTeReGo

                  My question is how are you "forcing" these gigabit interfaces to run at 10Mbit? Are these 10/100/1000 TX interfaces? Are you using the "speed 10" command to set these interfaces to 10Mbps? Or are these fiber links that you're using QoS or some other method to limit the bandwidth consumption to 10Mbps?

                    • Re: Avg Min/Max Charts exceeding port Speed

                      They are 10/100/1000 copper, not fiber, and yes forcing interfaces using "speed 10".   No QoS or other traffic shaping methods being done.  

                        • Re: Avg Min/Max Charts exceeding port Speed
                          aLTeReGo

                          Can you post a graph that shows what you're seeing?

                            • Re: Avg Min/Max Charts exceeding port Speed
                              Network_Guru

                              The results are dependent on the Orion polling interval being exact.
                              For instance, if you have interface statistics polling set to 300 seconds, but your poll is late and only gets to it after 330 seconds, then you can exceed the bandwidth utilization setting as the DB could have a higher number than is normally possible for a 300 second poll.
                              The DB will calculate the utilization based on the difference in interface counters over exactly 300 seconds.

                              Check your Polls per Second tuning to make sure you have the number of statistics polls per second set correctly.

                                • Re: Avg Min/Max Charts exceeding port Speed

                                   I see.  So if i understand correctly, Orion is using or applying it against the hard setting for interface polling and stat collection and not against the actual time of the received info.  That makes sense now.  I was thinking there was a function that took the delta of the octects and then also deltaed the time frame from last poll to figure out the per sec portion.  

                                  I have checked all my poller's tuning info and they are all keeping up.  All thresholds are above min requirements and CPU and mem load is low.  I know at times we miss a poll here and there but it is to be expected since I currently have over 12000+ interfaces most of which have a fairly high rate of polling. 

                                   
                                  Thank you for the info.

                                    • Re: Avg Min/Max Charts exceeding port Speed
                                      Network_Guru

                                      This is just an assumption on my part, but it seems to make sense.
                                      Only the SW application engineers would know for sure.....

                                        • Re: Avg Min/Max Charts exceeding port Speed
                                          Karlo.Zatylny

                                          In the poller, interface traffic for both in and out octets each keep their own timers.  Each time we detect that we have received a specific OID, we capture the time, calculate the time difference between the last time we saw this OID and use that for our BPS calculations.


                                          So now the issue is, why would you ever see a data rate above the possible setting of your NIC?  The issue probably comes in the summarizing of the data.  In the data table/view we store only one date time for all the columns, so there would be some inaccuracy between what we calculate in code versus what gets stored to the database.  The differences should hopefully never be huge but this methodology does leave some room for error.  Also, summarzing many points will lead to some more inaccuracy.  Min and Max values may be off slightly because of the first reason but shouldn't be affect by summarization.


                                          If your numbers are ridiculously off, then you may want to open a ticket and reference this thwack thread.  Unless you are satisfied that Orion is making your network perform beyond its capabilities :o)


                                          Thanks