How do you resolve missing mac addresses?

The simple answer is you need to frame your mind that UDT and IPAM can't actually source data from each other.  There is only the most minimal amount of crossover.  Makes more sense when you consider that they were both developed entirely separately so the devs couldn't assume that someone had both modules until someone in marketing decided to start pitching them as a bundle and then ultimately rolled them into the new licenses.   The way they are marketed and sold does not reflect the way they were built.  I think it's a good feature request for them to integrate the two much better, and with the way they are licensed now it really doesnt make sense for them to remain separate anymore, but thats also kind of a big project in terms of how both of them work at this time.  At the last company I had where we had both products I did some pretty heavy editing of the UDT endpoint views and got them where they would more clearly cross reference against all my other modules, but I don't have a copy of that handy to share anymore.  It was just a bunch of Custom Query widgets combined with me spending a lot of time crawling through the db tables piecing together whatever I could find that looked useful.

So getting to your number 2, yes neighbor scanning should help assuming all the pieces come together.  Neighbor scanning and automatic scanning arent really related.  Automatic scanning controls if/when the poller does a ping swing through the defined IP's and tracks if it gets responses or not.  Some people dont want all those ping sweeps all across their network, or they are documenting subnets that the poller wouldn't be able to reach so they disable it.

Without neighbor scanning IPAM is hoping it has SNMP access to your device. Frustratingly related to that lack of integration I mentioned this scan has to happen within IPAM, which has its own collection of SNMP creds that it can't just inherit from all the other places where you would have already defined them.  IPAM wants to do l2 and l3 discovery against every IP, even though NPM and UDT are also doing those exact same SNMP requests as well and writing the data to different tables. Neighbor scanning kicks in when you can't SNMP poll the device for some reason, but you have to specify the IP of a device that would have the mac address in the arp table.  Given that UDT works for these macs you know you can get this data, its just going to be a matter of making sure your config is all correct inside IPAM for it to put it all together.

https://support.solarwinds.com/SuccessCenter/s/article/How-IPAM-gets-a-MAC-Address-via-SNMP?language=en_US

https://documentation.solarwinds.com/en/success_center/ipam/content/ipam-neighbor-scanning.htm

Getting into the land of just hacking things up, in my past environment when I had discrepancies like this I once disabled all those kinds of topology polling in UDT and IPAM that all try the same OIDs and I used SQL queries to vacuum all the data I wanted out of the NPM topology tables and inserted it into the relevant tables in other modules.  Worked well for that environment and cut back on extra loads in my pollers and the target devices.

Thanks for your analysis. I have resorted to a lot of what you mention; e.g. routing through the tables to see what information I can extract from where. Part of my frustration comes from Solarwinds touting more integration between UDT and IPAM in the more recent releases, but this basic (in my mind, at least) functionality is still not present. And, yes; I do understand that these were different products, probably acquired and definitely developed separately. And I know that much of the work over the last three years is to work on getting all these different modules to share a common framework. But progress seems slow. I've been working with Solarwinds for a long time (15+ years?), and I'm not getting any younger. But I digress.

As I looked into this further, and particularly looked at the example screenshots I added originally, I see that I was mistaken about the subnet scan being the source of the mac address removals. Instead, I see two distinct events happening; a 'Neighbor scan' and a 'Subnet neighbor scan.' The 'subnet neighbor scan' seems to be recording the mac addresses, while the 'neighbor scan' seems to be removing them. 

The documentation (as we can all attest to) is frustratingly vague, but I assume these two items refer to the two different places one can enable neighbor scanning; e.g.

• The "global" subnet scan settings, where the default appears to be that 'SNMP neighbor scanning' is enabled (i.e. Configure subnet scan settings manually). I'm assuming that this is the 'Neighbor scanning' associated with my disappearing mac addresses (based on very little data).
• The "subnet" settings, where the default is to Disable Neighbor scanning (i.e. Neighbor scanning in IPAM). I assume this is the 'Subnet neighbor scanning' that is recording the mac addresses for me, however briefly.

So, we have two different settings that appear to do similar things. But the one at the subnet level is configurable, allowing you to set intervals and the IP address of the device to scan, while the other is simply on or off. And it appears that one giveth, and the other taketh away.

I know I'm not the first one to raise questions about how these two settings do (or do not) relate to one another (i.e. How does Neighbor Scanning work?), but I can't find any response that provides any clarification. So if anyone can shed some light on these two distinct (and possibly conflicting) settings, I'm sure we would all appreciate it.

In the meantime, I guess I'll keep experimenting. I may disable the global 'SNMP neighbor scanning' next and see that that does. Maybe I'll actually be able to retain those mac addresses in this particular subnet, but I'm concerned that I may lose something elsewhere.

So....

FYI. The 'Enable SNMP neighbor scanning' under the subnet scan settings is basically an on/off switch for all subnet neighbor scans. Disabling that setting disabled the neighbor scanning for any subnet where I had unset 'Disable Neighbor scanning' (nice double-negative in this case).

So, the essential logic is:

• To enable neighbor scanning for a subnet, the global 'Enable SNMP neighbor scanning' setting must be enabled and the 'Disable Neighbor scanning' setting must be disabled at the subnet level.
• To disable neighbor scanning for a subnet, you can either:
  • Leave the both the global and subnet settings enabled (essentially enable SNMP neighbor scanning globally, but Disable it for the subnet, which is the default)
    OR
  • Disable the global setting, which effectively tells IPAM to ignore where 'Disable Neighbor scanning' is disabled

So, now we know. 

The difference between IPAM and UDT is that the most basic information in IPAM is the IP Address while in UDT it would be the MAC address.

It seems you already sorted this out, but in order for UDT and IPAM work together, It needs to be able to gather and relate those information.

In IPAM, it needs to be able to see the MAC address on it's own.  In UDT, it also needs to be able to resolve to the Proper IP Address.

Once those information are populated on both Modules, that's when they can relate those information.