Skip to content

Fix Elastic IP association for instances with multiple network interfaces#194

Merged
aramprice merged 10 commits intocloudfoundry:masterfrom
neddp:fix_network_handling_with_multiple_nics
Mar 12, 2026
Merged

Fix Elastic IP association for instances with multiple network interfaces#194
aramprice merged 10 commits intocloudfoundry:masterfrom
neddp:fix_network_handling_with_multiple_nics

Conversation

@neddp
Copy link
Copy Markdown
Member

@neddp neddp commented Dec 12, 2025

Problem

When deploying BOSH instances with multiple network interfaces (using different nic_group configurations) and an Elastic IP, the CPI fails during create_vm with:

CPI error 'Unknown' with message 'There are multiple interfaces attached to instance 'i-xxxxx'. 
Please specify an interface ID for the operation instead.'

Additionally, there was no way to control which NIC receives the Elastic IP in multi-NIC configurations.

Root Cause

The associate_address AWS API call was using the instance_id parameter, which according to AWS documentation only works when "the instance must have exactly one attached network interface."

AWS Documentation Reference

From the AWS EC2 AssociateAddress API Documentation:

InstanceId parameter:

"The ID of the instance. The instance must have exactly one attached network interface. You can specify either the instance ID or the network interface ID, but not both."

NetworkInterfaceId parameter:

"The ID of the network interface. If the instance has more than one network interface, you must specify a network interface ID. You can specify either the instance ID or the network interface ID, but not both."

Solution

Multi-NIC Elastic IP fix:

  • Use network_interface_id instead of instance_id in associate_address call
  • Query instance network interfaces and target the correct NIC based on nic_group

nic_group support for VIP networks:

  • Users can specify which NIC receives the Elastic IP via nic_group in deployment manifest
  • Director passes nic_group to CPI, which maps it to the corresponding device_index
  • Backward compatible: defaults to primary NIC (device_index 0) when not specified

Additional improvements:

  • Added retry logic for AWS API calls to handle transient failures
  • Enhanced error messages with device_index and nic_group details

Depends on cloudfoundry/bosh#2658

@rkoster rkoster requested review from a team, mkocher and ragaskar and removed request for a team January 22, 2026 16:16
@rkoster rkoster moved this from Inbox to Pending Review | Discussion in Foundational Infrastructure Working Group Jan 22, 2026
aramprice
aramprice previously approved these changes Feb 26, 2026
@github-project-automation github-project-automation bot moved this from Pending Review | Discussion to Pending Merge | Prioritized in Foundational Infrastructure Working Group Feb 26, 2026
@beyhan beyhan requested a review from aramprice March 5, 2026 15:46
@beyhan beyhan moved this from Pending Merge | Prioritized to Pending Review | Discussion in Foundational Infrastructure Working Group Mar 5, 2026
@github-project-automation github-project-automation bot moved this from Pending Review | Discussion to Pending Merge | Prioritized in Foundational Infrastructure Working Group Mar 5, 2026
@aramprice aramprice merged commit 4ea69b8 into cloudfoundry:master Mar 12, 2026
2 checks passed
@github-project-automation github-project-automation bot moved this from Pending Merge | Prioritized to Done in Foundational Infrastructure Working Group Mar 12, 2026
Copy link
Copy Markdown

@ay901246 ay901246 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving two comments, one around deterministic indexing and the other around retries/exponential backoff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

6 participants