100,00 event limit not large enough...

I am getting close to 2k logs per second = 120k per minute or more.  The Historical search only allows a 100k limit over 1 minute...  Does this mean I will never be able to search all logs?  So far, that has been my experience.  Disappointing.

Parents
  • What type of information your are looking to retrieve?  I offset some daily routines by setup of reports to gather the data I need but it depend on the type of data you searching for.  I have used many programs each have  there own unique problems.  

    Prebuilt query are effective if there are very granular because 100K of event is not user friendly at all to a analysis.  

    As I mention I receive about 1-2K of events per minute put have no issues search week worth of logs for the information I need. The cap is there because 10GB memory for 100K of event means search a entire day will be over 100GB of memory which is not feasible.

Reply
  • What type of information your are looking to retrieve?  I offset some daily routines by setup of reports to gather the data I need but it depend on the type of data you searching for.  I have used many programs each have  there own unique problems.  

    Prebuilt query are effective if there are very granular because 100K of event is not user friendly at all to a analysis.  

    As I mention I receive about 1-2K of events per minute put have no issues search week worth of logs for the information I need. The cap is there because 10GB memory for 100K of event means search a entire day will be over 100GB of memory which is not feasible.

Children
No Data