This discussion has been locked. The information referenced herein may be inaccurate due to age, software updates, or external references.

You can no longer post new replies to this discussion. If you have a similar question you can start a new discussion in this forum.

Need help for SAM agents when same nodes are re-added.

bluefunelemental over 7 years ago

We are running into a real thorny situation where agent-monitored aws servers are being shutdown and re-built with a SAM agent in the image. Because the previous node is still in SAM the newly registered agent does not add itself as a node and requires manual intervention to both delete the previous node and redeploy agent to the new one.

While we are considering an automation where autoscale or ec2 deletion event triggers a node deletion from SAM but this is not our preferred solution due to the quick timing needed and loss of historical data.

Anyone else have a better solution?

Top Replies

stevenwhunt over 7 years ago +1

I'll dig into this and see what I can find on my side. Curious to see what others are doing in this area.

0 stevenwhunt over 7 years ago

I'll dig into this and see what I can find on my side. Curious to see what others are doing in this area.
Cancel
Vote Up +1 Vote Down

Cancel
0 bluefunelemental over 7 years ago in reply to stevenwhunt

stevenwhunt I think what we need is the ability for the agent manage as node process to see and take into account AWS instance-ID or if VMware maybe UUID. That way if one gets spun down and re-imaged then the new one will still be considered unique and as its an agent the overlapping IP is no issue.
In that case I would create a process on our side which unmanages a node when terminated so alerts and uptime is no longer affected but history is there. I'd also kick off an alert to ask engineering to delete when they no longer need that history.
Thoughts?
Cancel
Vote Up 0 Vote Down

Cancel
0 stevenwhunt over 7 years ago in reply to bluefunelemental

Good concept. I am looking into this scenario with some smarter people than I to see what is feasible today and what would have to be done longer term. Stay tuned.
Cancel
Vote Up 0 Vote Down

Cancel
0 stevenwhunt over 7 years ago in reply to bluefunelemental

Still digging on this, but I may have something that will work for you. Just want to verify functionality. By the way, I love this cloud stuff.
Cancel
Vote Up 0 Vote Down

Cancel
0 bluefunelemental over 7 years ago in reply to stevenwhunt

Great. One extra note - many of these have the same SID as they are just re-imaged from ami's and also many have a polling ip of 192.168.0.1 - maybe due to having multiple interfaces. That might get better once we apply some ip route tables and vrf's but for now I just switch the polling ip after it adds as node and re-discovers.
Thanks,
Christian
Cancel
Vote Up 0 Vote Down

Cancel