Modifying uplink's ipvX.ovn.ranges does not update logical routers' addresses #2907
Description
Is there an existing issue for this?
- There is no existing issue for this bug
Is this happening on an up to date version of Incus?
- This is happening on a supported version of Incus
Incus system details
config:
cluster.https_address: 192.168.201.11
core.bgp_address: 192.168.200.11
core.bgp_asn: "65100"
core.bgp_routerid: 192.168.200.11
core.https_address: :8443
network.ovn.ca_cert: |-
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
network.ovn.client_cert: |-
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
network.ovn.client_key: |-
-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----
network.ovn.northbound_connection: ssl:[192.168.201.11]:6641,ssl:[192.168.201.12]:6641,ssl:[192.168.201.13]:6641
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_stop_priority
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
- container_only_migration
- storage_zfs_clone_copy
- unix_device_rename
- storage_lvm_use_thinpool
- storage_rsync_bwlimit
- network_vxlan_interface
- storage_btrfs_mount_options
- entity_description
- image_force_refresh
- storage_lvm_lv_resizing
- id_map_base
- file_symlinks
- container_push_target
- network_vlan_physical
- storage_images_delete
- container_edit_metadata
- container_snapshot_stateful_migration
- storage_driver_ceph
- storage_ceph_user_name
- resource_limits
- storage_volatile_initial_source
- storage_ceph_force_osd_reuse
- storage_block_filesystem_btrfs
- resources
- kernel_limits
- storage_api_volume_rename
- network_sriov
- console
- restrict_dev_incus
- migration_pre_copy
- infiniband
- dev_incus_events
- proxy
- network_dhcp_gateway
- file_get_symlink
- network_leases
- unix_device_hotplug
- storage_api_local_volume_handling
- operation_description
- clustering
- event_lifecycle
- storage_api_remote_volume_handling
- nvidia_runtime
- container_mount_propagation
- container_backup
- dev_incus_images
- container_local_cross_pool_handling
- proxy_unix
- proxy_udp
- clustering_join
- proxy_tcp_udp_multi_port_handling
- network_state
- proxy_unix_dac_properties
- container_protection_delete
- unix_priv_drop
- pprof_http
- proxy_haproxy_protocol
- network_hwaddr
- proxy_nat
- network_nat_order
- container_full
- backup_compression
- nvidia_runtime_config
- storage_api_volume_snapshots
- storage_unmapped
- projects
- network_vxlan_ttl
- container_incremental_copy
- usb_optional_vendorid
- snapshot_scheduling
- snapshot_schedule_aliases
- container_copy_project
- clustering_server_address
- clustering_image_replication
- container_protection_shift
- snapshot_expiry
- container_backup_override_pool
- snapshot_expiry_creation
- network_leases_location
- resources_cpu_socket
- resources_gpu
- resources_numa
- kernel_features
- id_map_current
- event_location
- storage_api_remote_volume_snapshots
- network_nat_address
- container_nic_routes
- cluster_internal_copy
- seccomp_notify
- lxc_features
- container_nic_ipvlan
- network_vlan_sriov
- storage_cephfs
- container_nic_ipfilter
- resources_v2
- container_exec_user_group_cwd
- container_syscall_intercept
- container_disk_shift
- storage_shifted
- resources_infiniband
- daemon_storage
- instances
- image_types
- resources_disk_sata
- clustering_roles
- images_expiry
- resources_network_firmware
- backup_compression_algorithm
- ceph_data_pool_name
- container_syscall_intercept_mount
- compression_squashfs
- container_raw_mount
- container_nic_routed
- container_syscall_intercept_mount_fuse
- container_disk_ceph
- virtual-machines
- image_profiles
- clustering_architecture
- resources_disk_id
- storage_lvm_stripes
- vm_boot_priority
- unix_hotplug_devices
- api_filtering
- instance_nic_network
- clustering_sizing
- firewall_driver
- projects_limits
- container_syscall_intercept_hugetlbfs
- limits_hugepages
- container_nic_routed_gateway
- projects_restrictions
- custom_volume_snapshot_expiry
- volume_snapshot_scheduling
- trust_ca_certificates
- snapshot_disk_usage
- clustering_edit_roles
- container_nic_routed_host_address
- container_nic_ipvlan_gateway
- resources_usb_pci
- resources_cpu_threads_numa
- resources_cpu_core_die
- api_os
- container_nic_routed_host_table
- container_nic_ipvlan_host_table
- container_nic_ipvlan_mode
- resources_system
- images_push_relay
- network_dns_search
- container_nic_routed_limits
- instance_nic_bridged_vlan
- network_state_bond_bridge
- usedby_consistency
- custom_block_volumes
- clustering_failure_domains
- resources_gpu_mdev
- console_vga_type
- projects_limits_disk
- network_type_macvlan
- network_type_sriov
- container_syscall_intercept_bpf_devices
- network_type_ovn
- projects_networks
- projects_networks_restricted_uplinks
- custom_volume_backup
- backup_override_name
- storage_rsync_compression
- network_type_physical
- network_ovn_external_subnets
- network_ovn_nat
- network_ovn_external_routes_remove
- tpm_device_type
- storage_zfs_clone_copy_rebase
- gpu_mdev
- resources_pci_iommu
- resources_network_usb
- resources_disk_address
- network_physical_ovn_ingress_mode
- network_ovn_dhcp
- network_physical_routes_anycast
- projects_limits_instances
- network_state_vlan
- instance_nic_bridged_port_isolation
- instance_bulk_state_change
- network_gvrp
- instance_pool_move
- gpu_sriov
- pci_device_type
- storage_volume_state
- network_acl
- migration_stateful
- disk_state_quota
- storage_ceph_features
- projects_compression
- projects_images_remote_cache_expiry
- certificate_project
- network_ovn_acl
- projects_images_auto_update
- projects_restricted_cluster_target
- images_default_architecture
- network_ovn_acl_defaults
- gpu_mig
- project_usage
- network_bridge_acl
- warnings
- projects_restricted_backups_and_snapshots
- clustering_join_token
- clustering_description
- server_trusted_proxy
- clustering_update_cert
- storage_api_project
- server_instance_driver_operational
- server_supported_storage_drivers
- event_lifecycle_requestor_address
- resources_gpu_usb
- clustering_evacuation
- network_ovn_nat_address
- network_bgp
- network_forward
- custom_volume_refresh
- network_counters_errors_dropped
- metrics
- image_source_project
- clustering_config
- network_peer
- linux_sysctl
- network_dns
- ovn_nic_acceleration
- certificate_self_renewal
- instance_project_move
- storage_volume_project_move
- cloud_init
- network_dns_nat
- database_leader
- instance_all_projects
- clustering_groups
- ceph_rbd_du
- instance_get_full
- qemu_metrics
- gpu_mig_uuid
- event_project
- clustering_evacuation_live
- instance_allow_inconsistent_copy
- network_state_ovn
- storage_volume_api_filtering
- image_restrictions
- storage_zfs_export
- network_dns_records
- storage_zfs_reserve_space
- network_acl_log
- storage_zfs_blocksize
- metrics_cpu_seconds
- instance_snapshot_never
- certificate_token
- instance_nic_routed_neighbor_probe
- event_hub
- agent_nic_config
- projects_restricted_intercept
- metrics_authentication
- images_target_project
- images_all_projects
- cluster_migration_inconsistent_copy
- cluster_ovn_chassis
- container_syscall_intercept_sched_setscheduler
- storage_lvm_thinpool_metadata_size
- storage_volume_state_total
- instance_file_head
- instances_nic_host_name
- image_copy_profile
- container_syscall_intercept_sysinfo
- clustering_evacuation_mode
- resources_pci_vpd
- qemu_raw_conf
- storage_cephfs_fscache
- network_load_balancer
- vsock_api
- instance_ready_state
- network_bgp_holdtime
- storage_volumes_all_projects
- metrics_memory_oom_total
- storage_buckets
- storage_buckets_create_credentials
- metrics_cpu_effective_total
- projects_networks_restricted_access
- storage_buckets_local
- loki
- acme
- internal_metrics
- cluster_join_token_expiry
- remote_token_expiry
- init_preseed
- storage_volumes_created_at
- cpu_hotplug
- projects_networks_zones
- network_txqueuelen
- cluster_member_state
- instances_placement_scriptlet
- storage_pool_source_wipe
- zfs_block_mode
- instance_generation_id
- disk_io_cache
- amd_sev
- storage_pool_loop_resize
- migration_vm_live
- ovn_nic_nesting
- oidc
- network_ovn_l3only
- ovn_nic_acceleration_vdpa
- cluster_healing
- instances_state_total
- auth_user
- security_csm
- instances_rebuild
- numa_cpu_placement
- custom_volume_iso
- network_allocations
- zfs_delegate
- storage_api_remote_volume_snapshot_copy
- operations_get_query_all_projects
- metadata_configuration
- syslog_socket
- event_lifecycle_name_and_project
- instances_nic_limits_priority
- disk_initial_volume_configuration
- operation_wait
- image_restriction_privileged
- cluster_internal_custom_volume_copy
- disk_io_bus
- storage_cephfs_create_missing
- instance_move_config
- ovn_ssl_config
- certificate_description
- disk_io_bus_virtio_blk
- loki_config_instance
- instance_create_start
- clustering_evacuation_stop_options
- boot_host_shutdown_action
- agent_config_drive
- network_state_ovn_lr
- image_template_permissions
- storage_bucket_backup
- storage_lvm_cluster
- shared_custom_block_volumes
- auth_tls_jwt
- oidc_claim
- device_usb_serial
- numa_cpu_balanced
- image_restriction_nesting
- network_integrations
- instance_memory_swap_bytes
- network_bridge_external_create
- network_zones_all_projects
- storage_zfs_vdev
- container_migration_stateful
- profiles_all_projects
- instances_scriptlet_get_instances
- instances_scriptlet_get_cluster_members
- instances_scriptlet_get_project
- network_acl_stateless
- instance_state_started_at
- networks_all_projects
- network_acls_all_projects
- storage_buckets_all_projects
- resources_load
- instance_access
- project_access
- projects_force_delete
- resources_cpu_flags
- disk_io_bus_cache_filesystem
- instance_oci
- clustering_groups_config
- instances_lxcfs_per_instance
- clustering_groups_vm_cpu_definition
- disk_volume_subpath
- projects_limits_disk_pool
- network_ovn_isolated
- qemu_raw_qmp
- network_load_balancer_health_check
- oidc_scopes
- network_integrations_peer_name
- qemu_scriptlet
- instance_auto_restart
- storage_lvm_metadatasize
- ovn_nic_promiscuous
- ovn_nic_ip_address_none
- instances_state_os_info
- network_load_balancer_state
- instance_nic_macvlan_mode
- storage_lvm_cluster_create
- network_ovn_external_interfaces
- instances_scriptlet_get_instances_count
- cluster_rebalance
- custom_volume_refresh_exclude_older_snapshots
- storage_initial_owner
- storage_live_migration
- instance_console_screenshot
- image_import_alias
- authorization_scriptlet
- console_force
- network_ovn_state_addresses
- network_bridge_acl_devices
- instance_debug_memory
- init_preseed_storage_volumes
- init_preseed_profile_project
- instance_nic_routed_host_address
- instance_smbios11
- api_filtering_extended
- acme_dns01
- security_iommu
- network_ipv4_dhcp_routes
- network_state_ovn_ls
- network_dns_nameservers
- acme_http01_port
- network_ovn_ipv4_dhcp_expiry
- instance_state_cpu_time
- network_io_bus
- disk_io_bus_usb
- storage_driver_linstor
- instance_oci_entrypoint
- network_address_set
- server_logging
- network_forward_snat
- memory_hotplug
- instance_nic_routed_host_tables
- instance_publish_split
- init_preseed_certificates
- custom_volume_sftp
- network_ovn_external_nic_address
- network_physical_gateway_hwaddr
- backup_s3_upload
- snapshot_manual_expiry
- resources_cpu_address_sizes
- disk_attached
- limits_memory_hotplug
- disk_wwn
- server_logging_webhook
- storage_driver_truenas
- container_disk_tmpfs
- instance_limits_oom
- backup_override_config
- network_ovn_tunnels
- init_preseed_cluster_groups
- usb_attached
- backup_iso
- instance_systemd_credentials
- cluster_group_usedby
- bpf_token_delegation
- file_storage_volume
- network_hwaddr_pattern
- storage_volume_full
- storage_bucket_full
- device_pci_firmware
- resources_serial
- ovn_nic_limits
- storage_lvmcluster_qcow2
- oidc_allowed_subnets
- file_delete_force
- nic_sriov_select_ext
- network_zones_dns_contact
- nic_attached_connected
api_status: stable
api_version: "1.0"
auth: trusted
public: false
auth_methods:
- tls
auth_user_name: 9679d658257f443c41b063d877686d7c0ecb7a0a9a3621bacdd794718a00c48f
auth_user_method: tls
environment:
addresses:
- 192.168.201.11:8443
- 192.168.200.11:8443
- '[fd41:f666:2a1a:200:acc4:b4ff:fe4a:d041]:8443'
- '[2a02:1748:fc0c:609d:acc4:b4ff:fe4a:d041]:8443'
architectures:
- x86_64
- i686
certificate: |
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
certificate_fingerprint: ee536b48b9fce97a504ae3410838cddbd9a25377cc1b8da44013f8e08a911dec
driver: lxc | qemu
driver_version: 6.0.5 | 10.2.0
firewall: nftables
kernel: Linux
kernel_architecture: x86_64
kernel_features:
idmapped_mounts: "true"
netnsid_getifaddrs: "true"
seccomp_listener: "true"
seccomp_listener_continue: "true"
uevent_injection: "true"
unpriv_binfmt: "true"
unpriv_fscaps: "true"
kernel_version: 6.8.0-100-generic
lxc_features:
cgroup2: "true"
core_scheduling: "true"
devpts_fd: "true"
idmapped_mounts_v2: "true"
mount_injection_file: "true"
network_gateway_device_route: "true"
network_ipvlan: "true"
network_l2proxy: "true"
network_phys_macvlan_mtu: "true"
network_veth_router: "true"
pidfd: "true"
seccomp_allow_deny_syntax: "true"
seccomp_notify: "true"
seccomp_proxy_send_notify_fd: "true"
os_name: Ubuntu
os_version: "24.04"
project: default
server: incus
server_clustered: true
server_event_mode: full-mesh
server_name: node1
server_pid: 4326
server_version: "6.21"
storage: zfs | ceph
storage_version: 2.2.2-0ubuntu9.4 | 19.2.3
storage_supported_drivers:
- name: btrfs
version: 6.6.3
remote: false
- name: cephobject
version: 19.2.3
remote: true
- name: dir
version: "1"
remote: false
- name: truenas
version: 0.7.3
remote: true
- name: zfs
version: 2.2.2-0ubuntu9.4
remote: false
- name: ceph
version: 19.2.3
remote: true
- name: cephfs
version: 19.2.3
remote: true
- name: lvm
version: 2.03.16(2) (2022-05-18) / 1.02.185 (2022-05-18) / 4.48.0
remote: falseInstance details
No response
Instance log
No response
Current behavior
In a 3-node Incus cluster with functional OVN networking, due to network changes, I need to modify my ipv4.ovn.ranges and set ipv6.ovn.ranges (from previously unset). However, while changing the parameters via incus network edit or via incus network set is reflected in the Incus config, the respective logical routers' configuration is not updated in OVN.
There are no respective log events (warning, errors, etc) in /var/log/ovn/*.log either.
In my case, this basically disconnects the OVN networks from the physical network because I cannot perform the required address changes.
Expected behavior
Either editing ipvX.ovn.ranges is prevented (obviously not preferred), or changes actually propagate to OVN logical routers by renumbering their addresses.
Steps to reproduce
$ incus network show UPLINK
config:
bgp.peers.fw.address: 192.168.200.200
bgp.peers.fw.asn: "65100"
dns.nameservers: 10.0.0.53,fdd0:db2::53
ipv4.gateway: 192.168.254.1/24
[ ipv4.ovn.ranges: 192.168.254.129-192.168.254.190 ]
ipv4.routes: 10.0.0.0/8
ipv4.routes.anycast: "true"
ipv6.gateway: fd41:f666:2a1a:254::1/64
ipv6.routes: fdd0:db2::/32
ipv6.routes.anycast: "true"
mtu: "9000"
ovn.ingress_mode: routed
vlan: "300"
volatile.last_state.created: "true"
description: Physical network for OVN routers
name: UPLINK
type: physical
used_by:
- /1.0/networks/anycast-1?project=core-infra
- /1.0/networks/anycast-2?project=core-infra
- /1.0/networks/anycast-3?project=core-infra
- /1.0/networks/default?project=core-infra
managed: true
status: Created
locations:
- node1
- node2
- node3
project: default
$ incus network set UPLINK ipv6.gateway=2a02:fff:fff:609f::1/64 \
ipv6.ovn.ranges=2a02:fff:fff:609f:f000::100-2a02:fff:fff:609f:f000::200
$ incus network show UPLINK
config:
bgp.peers.fw.address: 192.168.200.200
bgp.peers.fw.asn: "65100"
dns.nameservers: 10.0.0.53,fdd0:db2::53
ipv4.gateway: 192.168.254.1/24
ipv4.ovn.ranges: 192.168.254.129-192.168.254.190
ipv4.routes: 10.0.0.0/8
ipv4.routes.anycast: "true"
[ ipv6.gateway: 2a02:fff:fff:609f::1/64 ]
[ ipv6.ovn.ranges: 2a02:fff:fff:609f:f000::100-2a02:fff:fff:609f:f000::200 ]
ipv6.routes: fdd0:db2::/32
ipv6.routes.anycast: "true"
mtu: "9000"
ovn.ingress_mode: routed
vlan: "300"
volatile.last_state.created: "true"
description: Physical network for OVN routers
name: UPLINK
type: physical
used_by:
...
$ incus network info anycast-1
Name: anycast-1
MAC address: 10:66:6a:5f:d5:50
MTU: 1500
State: up
Type: broadcast
IP addresses:
inet 10.10.11.1/24 (link)
inet6 fdd0:db2:10:11::1/64 (link)
OVN:
Chassis: node3
Logical router: incus-net10-lr
Logical switch: incus-net10-ls-int
IPv4 uplink address: 192.168.254.130
[ IPv6 uplink address: fd41:f666:2a1a:254:1266:6aff:fe5f:d550 ] still showing previous address
$ ovn-nbctl show
...
router cb228150-0c2f-4ffb-bd46-3a5bb93aaac8 (incus-net10-lr)
port incus-net10-lr-lrp-ext
mac: "10:66:6a:5f:d5:50"
networks: ["192.168.254.130/24", "fd41:f666:2a1a:254:1266:6aff:fe5f:d550/64"]
port incus-net10-lr-lrp-int
mac: "10:66:6a:5f:d5:50"
networks: ["10.10.11.1/24", "fdd0:db2:10:11::1/64"]
...
Same when changing ipv4.ovn.ranges which is not being propagated to the OVN logical routers.
Is there a way to manually trigger a re-configuration on OVN-side or am I missing something? Could not find this documented anywhere, nor in the forum.
Metadata
Metadata
Assignees
Labels
No labels