• Odd interaction between /proc/sys/net/ipv6/conf/all/forwarding & /sbin

    From Andrew Gideon@110:110/2002 to All on Mon Dec 9 20:28:49 2013
    Subject: Odd interaction between /proc/sys/net/ipv6/conf/all/forwarding &
    /sbin/ip -6 addr add

    I want to add an IP to my machine which is essentially
    one side of a point-to-point Ethernet link. I use:

    /sbin/ip -6 addr add local 2606:1a00:0:5000::527/127 dev eth0.341

    When /proc/sys/net/ipv6/conf/all/forwarding is 0, this
    is fine. Among other effects, this adds to the output of:

    /sbin/ip -6 route show table local

    the entry:

    local 2606:1a00:0:5000::527 via :: dev lo proto none metric 0 mtu 16436 advmss 16376 hoplimit 0

    and connectivity works.

    However, if /proc/sys/net/ipv6/conf/all/forwarding is 1, then the
    same "/sbin/ip -6 addr add ..." causes to be added to the "local"
    table:

    local 2606:1a00:0:5000::526 via :: dev lo proto none metric 0 mtu 16436 advmss 16376 hoplimit 0
    local 2606:1a00:0:5000::527 via :: dev lo proto none metric 0 mtu 16436 advmss 16376 hoplimit 0

    Note that 2606:1a00:0:5000::526 is the remote side of the point-to-point
    link. With the entry in the "local" table routing traffic to that IP to
    the loopback device, connectivity fails. That's what makes this a problem.

    If I reverse the order of events - if I add the address while forwarding
    is set to zero and then enable forwarding, the single entry in the
    "local" table becomes the two entries once forwarding is enabled.

    As I mentioned, having the remote IP mapped to loopback breaks connectivity.
    If I reach this problematic state and then:

    /sbin/ip route delete 2606:1a00:0:5000::526 dev lo table local

    then the remote IP is again reachable. So I have a workaround.

    But this seems ridiculous! Clearly I'm doing something wrong, but I cannot guess what that might be.

    Any suggestions would be welcome.

    Thanks...

    Andrew

    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: UsenetServer - www.usenetserver.com (110:110/2002@linuxnet)
  • From Richard Kettlewell@110:110/2002 to All on Mon Dec 9 21:22:10 2013
    Subject: Re: Odd interaction between /proc/sys/net/ipv6/conf/all/forwarding & /sbin/ip -6 addr add

    Andrew Gideon <c182driver9@gideon.org> writes:
    However, if /proc/sys/net/ipv6/conf/all/forwarding is 1, then the
    same "/sbin/ip -6 addr add ..." causes to be added to the "local"
    table:

    local 2606:1a00:0:5000::526 via :: dev lo proto none metric 0 mtu 16436
    advmss 16376 hoplimit 0
    local 2606:1a00:0:5000::527 via :: dev lo proto none metric 0 mtu 16436
    advmss 16376 hoplimit 0

    Note that 2606:1a00:0:5000::526 is the remote side of the point-to-point link. With the entry in the "local" table routing traffic to that IP to
    the loopback device, connectivity fails. That's what makes this a
    problem.

    https://tools.ietf.org/html/rfc4291#section-2.6.1

    --
    http://www.greenend.org.uk/rjk/

    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: Anjou (110:110/2002@linuxnet)
  • From Andrew Gideon@110:110/2002 to All on Tue Dec 10 03:04:54 2013
    Subject: Re: Odd interaction between /proc/sys/net/ipv6/conf/all/forwarding &
    /sbin/ip -6 addr add

    On Mon, 09 Dec 2013 21:22:10 +0000, Richard Kettlewell wrote:

    Andrew Gideon <c182driver9@gideon.org> writes:
    However, if /proc/sys/net/ipv6/conf/all/forwarding is 1, then the
    same "/sbin/ip -6 addr add ..." causes to be added to the "local"
    table:

    local 2606:1a00:0:5000::526 via :: dev lo proto none metric 0 mtu 16436 advmss 16376 hoplimit 0
    local 2606:1a00:0:5000::527 via :: dev lo proto none metric 0 mtu 16436 advmss 16376 hoplimit 0

    Note that 2606:1a00:0:5000::526 is the remote side of the point-to-point
    link. With the entry in the "local" table routing traffic to that IP to
    the loopback device, connectivity fails. That's what makes this a
    problem.

    https://tools.ietf.org/html/rfc4291#section-2.6.1

    Thanks.

    I have confirmed that this additional route occurs on the
    "zero" of the subnet regardless of the prefix size, and that
    it is occurring only if forwarding is enabled.

    What I'm finding most interesting about this is that two
    different upstreams have assigned /127s for the "point to
    point" used for BGP peering. This suggests that they both
    erred in the same fashion.

    Still, it is claimed in http://packetlife.net/blog/2010/may/6/ipv6-127-prefixes/
    that Cisco seems to ignore the subnet-router anycast on
    a /127. More recently, there's RFC6164. Does Linux support
    "Routers MUST support the assignment of /127 prefixes on point-to-point inter-router links" - from the recommendations section?

    I did find http://comments.gmane.org/gmane.linux.network/200180

    I've more reading to do, I see.

    - Andrew


    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: UsenetServer - www.usenetserver.com (110:110/2002@linuxnet)
  • From Pascal Hambourg@110:110/2002 to All on Tue Dec 10 08:47:55 2013
    Subject: Re: Odd interaction between /proc/sys/net/ipv6/conf/all/forwarding
    & /sbin/ip -6 addr add
    Reply-To: pascal.news@plouf.fr.eu.org

    Andrew Gideon a ‚crit :
    More recently, there's RFC6164. Does Linux support
    "Routers MUST support the assignment of /127 prefixes on point-to-point inter-router links" - from the recommendations section?

    Yes, since Linux 3.1.

    commit 2bda8a0c8af5294b869da1efd2c2b9a562f50dcf
    Author: Bj<C3><B8>rn Mork <bjorn@mork.no>
    Date: Tue Jul 5 23:04:13 2011 +0000

    Disable router anycast address for /127 prefixes

    RFC 6164 requires that routers MUST disable Subnet-Router anycast
    for the prefix when /127 prefixes are used.

    No need for matching code in addrconf_leave_anycast() as it
    will silently ignore any attempt to leave an unknown anycast
    address.

    commit 32019e651c6fcee6bad6b6adb171feea5af0d16b
    Author: YOSHIFUJI Hideaki <yoshfuji@linux-ipv6.org>
    Date: Sun Jul 24 11:44:34 2011 +0000

    ipv6: Do not leave router anycast address for /127 prefixes.

    Original commit 2bda8a0c8af... "Disable router anycast
    address for /127 prefixes" says:

    | No need for matching code in addrconf_leave_anycast() as it
    | will silently ignore any attempt to leave an unknown anycast
    | address.

    After analysis, because 1) we may add two or more prefixes on the
    same interface, or 2)user may have manually joined that anycast,
    we may hit chances to have anycast address which as if we had
    generated one by /127 prefix and we should not leave from subnet-
    router anycast address unconditionally.

    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: Plouf ! (110:110/2002@linuxnet)
  • From Richard Kettlewell@110:110/2002 to All on Tue Dec 10 08:56:51 2013
    Subject: Re: Odd interaction between /proc/sys/net/ipv6/conf/all/forwarding & /sbin/ip -6 addr add

    Andrew Gideon <c182driver9@gideon.org> writes:
    Still, it is claimed in http://packetlife.net/blog/2010/may/6/ipv6-127-prefixes/ that Cisco
    seems to ignore the subnet-router anycast on a /127. More recently,
    there's RFC6164. Does Linux support "Routers MUST support the
    assignment of /127 prefixes on point-to-point inter-router links" -
    from the recommendations section?

    I did find http://comments.gmane.org/gmane.linux.network/200180

    I've more reading to do, I see.

    If I’m reading the history right, the change described there should be present in kernel 3.1 and later. ICBW. What kernel do you see this
    behavior on?

    --
    http://www.greenend.org.uk/rjk/

    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: Anjou (110:110/2002@linuxnet)
  • From Andrew Gideon@110:110/2002 to All on Tue Dec 10 14:22:42 2013
    Subject: Re: Odd interaction between /proc/sys/net/ipv6/conf/all/forwarding &
    /sbin/ip -6 addr add

    On Tue, 10 Dec 2013 08:56:51 +0000, Richard Kettlewell wrote:

    If I’m reading the history right, the change described there should be present in kernel 3.1 and later. ICBW.

    Sure. I've a workaround, and - if I didn't - I could always request a different network for the peering (though I suppose I could be told "no";
    one of those upstreams is very procedure-laden).

    I am somewhat surprised, though, that it hasn't been backported. I
    cannot imagine that this situation is all that unusual. Perhaps people
    using Linux for routing are more aware than I, and are better about
    getting more than a /127 for the peering from their peers.

    Mostly, though, I'm glad to now understand this. Mysteries worry me.

    What kernel do you see this
    behavior on?

    2.6.32-431.el6.i686 on CentOS.

    Thanks...

    Andrew

    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: UsenetServer - www.usenetserver.com (110:110/2002@linuxnet)
  • From Richard Kettlewell@110:110/2002 to All on Tue Dec 10 17:01:27 2013
    Subject: Re: Odd interaction between /proc/sys/net/ipv6/conf/all/forwarding & /sbin/ip -6 addr add

    Andrew Gideon <c182driver9@gideon.org> writes:
    I am somewhat surprised, though, that it hasn't been backported. I
    cannot imagine that this situation is all that unusual. Perhaps people using Linux for routing are more aware than I, and are better about
    getting more than a /127 for the peering from their peers.

    I can’t speak for CentOS’s backporting policies. Howeve the stable
    kernel series left 2.6.32 behind some long time ago, and the policy
    summarizes itself as “in short, something critical” which seems like a higher bar, though I don’t know how broadly critical gets interpreted.

    Mostly, though, I'm glad to now understand this. Mysteries worry me.

    l-)

    --
    http://www.greenend.org.uk/rjk/

    --- MBSE BBS v1.0.1 (GNU/Linux-i386)
    * Origin: Anjou (110:110/2002@linuxnet)