16 Replies Latest reply on Jan 29, 2013 2:09 PM by Lawrence Garvin

    Patch Manager -  Federated model through firewalls and with NAT

    the_toilet

      Hi forum

       

      Does anyone else find this product to be completely non-intuitive in a federated install?  i am really having trouble with it, and am ready to throw it out completely and get something else.  now to my questions....

       

      Has anyone managed to get this software working through lots of firewalls, and with NAT?  scenario:

       

           Primary patch manager server is in my data centre - Local IP address 10.10.10.10

           Secondary patch manager server (all roles) on customer site - Local IP Address 20.20.20.20

           WSUS server on customer site - Local IP Address 20.20.20.21

       

      As there is network address translation between the PM server, and firewalls of course, they appear as diferent IP Addresses to each other:

       

           The secondary PM server appears to the primary server as 10.55.20.20, and the WSUS server appears as 10.55.20.21

       

      what this means is, when i try to configure anything at the parent end, like try to plug into the WSUS serve ronsite, i fdo not seem to be able to tell it what remote PM server to use (like you would if this were the nice easy storage0-managewr product)

       

      am i missing something?  when i try to connetc to the customer server running PM i can not log in as the users have no permissions for anything...  i think i am just stuck at stage one, even though i have been playing and configuring for months..  i am still yet to see a patch in this product LOL

        • Re: Patch Manager -  Federated model through firewalls and with NAT
          Lawrence Garvin

          Greetings Dan

           

          A number of customers have Patch Manager working across firewalls. It's design was optimized for that purpose.

          However, NAT is always problematic for any sort of application.

          The first question I need to ask is exactly how you are facilitating the connection between the primary server on your network and any of the secondary servers on customer sites?

          Do you have a site-to-site VPN?

          Are you using a system-to-site VPN?

          Are the secondary servers published as Internet-accessible systems on the customer's firewall?

           

          Where is the 10.55 network coming from?

           

          You'll definitely need to create User accounts on the customer's secondary server, as each application server has its own set of authorized users and credential rings. At a minimum the Patch Manager local Administrator account will always provide this capability. Beyond that, a standard account for your use could be created whenever a secondary server is deployed to a customer site -- this could be a domain account in the customer's domain, or a local account on the Patch Manager server.

            • Re: Patch Manager -  Federated model through firewalls and with NAT
              the_toilet

              Hi LGarvin,

              sorry for the delay...  i have been working on toehr stuff and now as always, this one is urgent again...  i am hoping ot get Patch Manager working so i can do a report on a customer estate showing windows version and patch levels across the estate.

               

              of course, i have hit all the same problems as before, and have yet to get a task to run ont he remote application server

               

              so, you questions:

               

              1. it is a MPLS tail into the custopmers MPLS network.  we have firewalls either end of this tail.  NAT'ing is only done at our end of the firewalls

               

              None of the patch management infrastructure is internet facing

               

              example of ip[ addressing:

               

              PAS              =      local IP     =     10.2.2.10

              remote AS     =     Local IP     =     10.1.6.50

               

              remote AS is seen from the PASas 10.55.6.50

              PAS is seen from remote AS as 10.2.201.16

               

              WSUS server sits ont he customer network with a local IP, the PAS has zero network visibility of that customer WSUS server

               

               

              problems so far:

               

              1. no tasks will run on the remote AS, they all sit in running

              2. no discovery or inventory has ever ran

              3. you do not seem to be able to make any significant changes ont he AS, they all have to be ont he PAS, even though i was hoping to delegate authoroty to an onsite team who woudl use the remote AS rather than use the PAS

              4. how do you add a remote windows domain to the remote-AS, when you have to add it to the PAS first whi can not see it?

               

              i am sure i am missing something simple, otherwise this is not a federated product as sold....

               

              if i try to do anything on the remote-AS around adding users to the patch manager groups, i get:

               

               

              Source: Csla

               

              Exception occurred at 23/01/2013 08:36:42: The submitted task cannot be executed because the requesting user's credential ring does not contain any credentials.

               

              Please use the credential ring wizard and assign at least one credential

               

              DataPortal_Execute method call failed

               

              DataPortal.Update failed

               

               


                • Re: Patch Manager -  Federated model through firewalls and with NAT
                  Lawrence Garvin

                  A couple of things may be happening with the task executions. Ensure that you've configured the necessary Automation Server Routing Rules, so that the Automation Role server on the remote AS is only managing tasks for systems in the local subnet (e.g. 10.1.6/24) and likewise for the PAS, that it's AutoServer is only managing tasks for systems in the PAS subnet (10.2.2/24).

                   

                  Are you saying that the tasks are scheduled, but not executing? There may also be a communications issue because of the NAT. The AS registration includes an IP Address, but the IP Address in the registration may not be the same one that the PAS actually needs to communicate with. My understanding is that we do first a DNS lookup on the hostname, and if that fails, we fallback to the stored IP Address. Verify that the DNS on the PAS network properly resolves the hostname of the remote AS to the 10.55.6.50 address.

                   

                  All communication in a Patch Manager environment is always downstream PAS -> SAS; AppServer -> MgmtServer; MgmtServer -> AutoServer; AutoServer -> Client. One thing to keep in mind, though, is that if the remote AS is not also a Management Server for that customer site, then the remote AS will need to communicate with the Management Server hosted on the PAS. Ensure that port 4092 is open bi-directionally.

                   

                  Management objects -- e.g. domains, workgroups, WSUS servers, -- what Patch Manager identifies as "scopes" must first be defined on the PAS and then those objects will replicate to the remote AS. Security credentials, Security Role memberships, and User Profiles are all local to each AppServer. Once you create the scope objects (e.g. the customer's WSUS server and DOMAIN), you absolutely can delegate access/management of those scopes to accounts exclusive to the remote AS. Once the scopes are added on the PAS and replicated to the remote AS, the User Preferences settings allow you to 'hide' those scopes from individual users. Hiding the scopes is merely a matter of visual convenience, because without an actual credential that can authenticate with that scope, it cannot be accessed. Chapters 8 and 9 of the Administration Guide may be helpful to some extent, and beyond that I'm happy to help you work through some of the more intricate details.

                  1 of 1 people found this helpful
              • Re: Patch Manager -  Federated model through firewalls and with NAT
                the_toilet

                thanks for all the advice...    understanding that the traffic is all one direction helps a lot actually, thank you

                 

                I have managed to get the system talking to the customer WSUS server...  like you aid, all the work is configured on the PAS but then you go onto the remote AS to actually browse and what not

                 

                the next issue is, i can not get the WMI agent to go out to ANY servers at all, no matter what i try.  i had to disable auto-roll out of the agent as it is a customer site and i do not know what the efects are and it is a very paranoid customer.  because of this, i have still NOT managed ot get even one inventory task to run...

                 

                i created a task to push out the agent to 4 test servers...  the task completed but when i check the content, NONE of the agents rolled out...  how do i debug this?  One of the reasons that i am using patch manager, is for inventory.  i tried to use microsots MBSA product to scan the domain, but it failed too in that it could not talk to the remote servers WSUS agent...  i am worried that this problem in Patch Manager is the same issue...

                 

                there are no firewalls, no vlans, and no windows firewalls...  what can i do?  getting ready to throw this product out and ask for a refund

                  • Re: Patch Manager -  Federated model through firewalls and with NAT
                    Lawrence Garvin

                    I'll make an educated guess that the likely roadblock to deploying the WMI Providers in your scenario will be the need to access file sharing, specifically we use the ADMIN$ share on the target system. An Automation Role server on the customer's site will help tremendously with this, as then the file share connection comes from a local server, rather than thru the firewall. (Nobody wants to open port 445 on a firewall. )

                     

                    The WMI Providers are required to perform a Managed Computer Inventory.

                     

                    Also.. with respect to computer inventories... you might also consider using the WSUS Extended Inventory as an alternative to WMI-based Managed Computer Inventory. The WSUS Extended Inventory is a configuration option enabled in the Patch Manager console, on a per-WSUS-server basis, and leverages a feature designed in WSUS for System Center Essentials. The WSUS Extended Inventory triggers the WUAgent to upload asset inventory data to the WSUS server, where it is stored in the WSUS database. This information is then captured by Patch Manager via the WSUS Inventory, from the WSUS server, rather than having to launch an individual RPC/WMI-based connection to each system in the enterprise.

                     

                    The disadvantage to the WSUS Extended Inventory is that you have no granular control over what is collected, or when it is collected. "What?" == "everything the WUAgent is configured to collect" and "When?" == "anytime the WUAgent performs a WSUS detection/reporting event". This inventory data can be reported on via a set of reports in the Windows Server Update Services report category that are prefixed with the title string "WSUS Inventory - ". Those reports can also be consolidated, via report customization.

                     

                    To enable WSUS Extended Inventory for a WSUS server, right click on the WSUS node in the Patch Manager console, select the "Configuration Options" menu item, and enable "Collect Extended Inventory Information" -- the first item in the dialog.

                      • Re: Patch Manager -  Federated model through firewalls and with NAT
                        the_toilet

                        Hi there

                         

                        thank you for the reply... i just tried to enable this featuere, and got my next error message  :-(.  i can browse the structure int he WSUS server, so it can talk to it, but this is wehat i get whenever i try to do anything with it  :-(

                         

                        error:

                         

                         

                        Source: EminentWare.Core.Client.RPC

                        Exception occurred at 28/01/2013 09:08:22: Communication failure.

                        The RPC protocol sequence is not supported.

                        Details: dgsrpcinterface::smartbind() failed.

                          • Re: Patch Manager -  Federated model through firewalls and with NAT
                            Lawrence Garvin

                            Browsing the WSUS node in the console and approving updates are all done via the WSUS API, via the webserver.

                             

                            However, performing other tasks are done via WMI, and like any other client need to have RPC (135), SMB (445) and WMI open to faciliate that. SMB is handled easily by ensuring that File & Print Sharing are enabled. (File Sharing is required to publish third party updates, so this needs to be enabled anyway, and should be, by default).

                             

                            However, on Vista and later systems, for some unexplicable reason, Microsoft left the Windows Management Instrumentation (WMI) ruleset disabled, by default, in the Windows Firewall. (Maybe because WMI requires RPC, is my best guess).

                             

                            At any rate, you'll also need to enable all three "Windows Management Instrumentation" rules in the Advanced Firewall configuration.

                            Also, verify that RPC/DCOM rules are enabled for access to RPC on port 135.

                              • Re: Patch Manager -  Federated model through firewalls and with NAT
                                the_toilet

                                Hi there

                                 

                                thanks again for the help...  just checked and confirmed that there is no firewall running on the WSUS server, and no firewlal running on the customer-network located AS.  there is nothing betweee them ont he LAN either, so we should be all set unless there are servcies not running that i need to check on the WSUS server?

                                 

                                with regard to "To enable WSUS Extended Inventory for a WSUS server".  i have done this now (at least i think i have).  when i tried to schedule the enablement, i got the error described above, but when i told it to do it NOW, it seemed to take the setting...  not sure how we verify if it is actually working.  how long should it be, before one of the WSUS inventory reports show some data?

                                 

                                i think there is magic going on in this customer network as i have no idea how the servers were built in terms of GPO etc, maybe there are some things not working that should.  i have test servers so i can test anything you suggest...

                                 

                                i got a new problem today where i can no longer browse the Domain, even though i could once...  it not gives a referal error

                                 

                                    "Unable to connect to the resource using the account Reason: A referral was returned from the server."

                                  • Re: Patch Manager -  Federated model through firewalls and with NAT
                                    Lawrence Garvin

                                    Enabling the "Collect Extended Inventory Information" option is an on-demand configuration change; it shouldn't have offered you the ability to 'schedule' that change. I'll check into that behavior. You can confirm its working in two different ways:

                                    1. The WSUS Inventory group of reports will be populated with data after the next WSUS Inventory task is executed. The WSUS Inventory task will be the key event in making the reporting data available in Patch Manager.

                                    1. In the WindowsUpdate.log there will be additional entries for the inventory upload. You can force this to occur on any client by launching a detection event: wuauclt /detectnow. Check back in about 20-30 minutes and you should see confirmation in the logfile. (Note: The DetectNow can also be launched from the Patch Manager console.)

                                     

                                    I'll inquire about the "referral" message. I have not previously encountered that.

                                      • Re: Patch Manager -  Federated model through firewalls and with NAT
                                        the_toilet

                                        hey, i manually kicked off a WSUS inventory, and this time it is working, i am trying to look back at what i have changed to make it happen...  i have rebooted the patch manager server, and also got the RAM upgrade to 8GB

                                         

                                        the inventory seems ot be going through each and ever patch, and each and ever devcie within WSUS...  does that sound right?  will it do this each time the inventory is ran or will it only look for diferences?

                                         

                                        by the way, i really appreciate this help, you are a gold mine of information

                                          • Re: Patch Manager -  Federated model through firewalls and with NAT
                                            Lawrence Garvin

                                            The inventory tasks are always full-scale data collection efforts. With the WMI Managed Computer Inventory you have the ability to choose what you collect, and when (or how often) you collected it, but it has a dependency on RPC/WMI. The WSUS Extended Inventory is an all-or-nothing proposition. All clients, all data points, every day.

                                             

                                            Glad to hear the conversation has been helpful to you; that's why I'm here. :-)

                                              • Re: Patch Manager -  Federated model through firewalls and with NAT
                                                the_toilet

                                                hmm..  then i need to work on trying to find why i can not get the WIM provider pushed out...  i can not even get it installed onto the WSUS server, so i am not sure what is happening...  i will try a few of the options like asking the AS server to set the admin$ share

                                                  • Re: Patch Manager -  Federated model through firewalls and with NAT
                                                    the_toilet


                                                    sorry, just found another issue....  i made 2 new reports last night, name "_dan - blah blah blah desktops" and one for servers.  i exported one and still have the PDF, but when i go in today the reports are gone...  any thoughts on why this might happen?

                                                      • Re: Patch Manager -  Federated model through firewalls and with NAT
                                                        Lawrence Garvin

                                                        Not any immediate thoughts; I've not heard of this type issue before. It's fundamentally impossible to not save a custom report (except cancelling from the dialog, which would have prevented it's generation), and its absolutely impossible to overwrite a report provided by Patch Manager.

                                                         

                                                        One possibility though.... there are two types of reports "System" and "User Defined". As I recall, the report listings, by default, are sorted by report type, pushing the "User Defined" reports to the bottom of the list, not necessarily in alphabetical order with the "System" Reports.

                                                         

                                                        One thing I've found useful is to group my report listing views by report type, and then I can quickly drill down into the "User Defined" group to find my custom reports.

                                                        Another variation on this, or used in conjunction with, is to sort descending by "Date Modified", so that the newest report definitions are always at the top of the listing.

                                                          • Re: Patch Manager -  Federated model through firewalls and with NAT
                                                            the_toilet

                                                            thanks for the reply..  sorted by all methods and they have gone, very very odd...  i have been deleting scheduled tasks today to try and clean the system up a bit, but not been in thr reports area

                                                             

                                                            i know what you mean about over-writione, as the application tells you that you alread have a report and lets you rename it...  i actually made a report to show workstations, and then saved it, re-ran it for servers and saved it with a new name...

                                                             

                                                            both gone...  i know i have nbot gone crazy as i still have the PDF export of the report when ran called "_Dan - Servers with approved update percentages.pdf"

                                                             

                                                            its mad how much work this product is to make it work... i am so used to other solarwinds products where you have them doing the basics within 30 mins...  i am 2 months into patch manager now, and still not where i need to be..  have learnt a lot though, but i think the cost of ownership is a lot higher than planned LOL

                                                              • Re: Patch Manager -  Federated model through firewalls and with NAT
                                                                Lawrence Garvin

                                                                I understand your frustrations, Dan.

                                                                 

                                                                Patch Manager is an 'acquired' product, and had four years of extensive development done on it prior to SolarWinds' acquisition last year. The focus of the previous development cycles was on feature-functionality, and not on user experience -- unlike the SolarWinds products which were built from the ground-up with the user experience in mind, and are a hallmark of the SolarWinds brand.

                                                                 

                                                                Rest assured, though, that a lot of attention is being paid to the usability surrounding Patch Manager. The v1.8 release introduced a number of usability enhancements, and as announced here just a few days ago, there are more in the works.

                                                                 

                                                                Although, to be fair, a multi-tenant MSP installation is an advanced installation of Patch Manager, and there's only so much you can do with wizards.