John Vossler wrote:
> Greetings,
>
> I have a new system running Solaris 10 set up as an NTP server. IT is
> synchronizing correctly but I cannot get it to multicast on any
> interface except the systems primary Ethernet interface, bge0. I need
> it to multicast on interfaces bge1-bge3 and ce0 - ce7.
>
> Client systems reside on the network segments on these other interfaces.
>
> Anyone have any suggestions on getting the server to multicast on these
> other interfaces?
>
> John

Are you talking about outgoing packets to a multicast address? I think
that there may be some bugs in that area since the addresses are limited
and choosing an outgoing address becomes an issue and it will only send
it out on one address. I'm not sure if it's possible to set up a server
(at least not easily) to send via different interfaces.

To do this correctly you need to be able to set up a different multicast
address to use and associate it with a particular server address for
each interface. The only way that I can think of to accomplish this is
to be able to specify a specific binding address in the config file. We
would have to add that as an optional parameter to the broadcast directive.

Danny

John Vossler wrote:

> Danny,
>
> You are correct, I am talking about outgoing multicast intended for
> client systems in our environment.
>
> I not certain I completely understand what you mean about associating
> interfaces with multicast addresses. It seems like you are inferring
> that I would need to define a unique multicast address for each
> interface on the NTP server. Am I understanding this correctly?
>

Yes.

> My original intent was to have the NTP server multicast 224.0.1.1 on all
> 12 interfaces, and have the clients all listen for the same multicast
> address.
>

I don't believe it can be done that way but I'd have to check the
multicast protocol to be sure. As far as I know you have to send out
multiple packets, one for each interface, and you have to do that
explicitly. You certainly cannot do that today in NTP since it only uses
one interface to send out to the multicast address.

> My ntp.conf file on the server just has the single broadcast line
> "broadcast 224.0.1.1 ttl 4" it does not appear to attached to any
> specific interface; thus your comment about adding a parameter.
>

Correct. That does not exist today.

> My initial reaction to this issue was to determine if there was an OS
> (Solaris) directive to include the multicast for other interfaces in the
> group as reported by "netstat -g". Or a method to specify the multicast
> interfaces in the ntp.conf file (apparently not, or not easily).
>

No, and certainly it has not been implemented in NTP.

> Is it possible to accomplish what I intended? Or is there a method to
> specify the single interface that ntp is multicasting on? I might be
> able to get the environment in sync if I could multicast on just ce3
> instead of bge0; the interface it is using currently.
>

We'd have to add that to the code. It does not exist today. Please
submit a bug report for an enhancement to add the option so it won't get
forgotten.

> One note. I am running the ntp code shipped with Solaris 10 which is v3.
>
> John
>

We cannot do anything with V3 and we would certainly not touch that
code. V3 was retired a long time ago. Upgrade to V42.4 at least, you
won't be sorry.

Danny

Danny Mayer wrote:
> John Vossler wrote:
[...]
>> My initial reaction to this issue was to determine if there was an OS
>> (Solaris) directive to include the multicast for other interfaces in the
>> group as reported by "netstat -g". Or a method to specify the multicast
>> interfaces in the ntp.conf file (apparently not, or not easily).
>
> No, and certainly it has not been implemented in NTP.

If ntpd has been configured to send multicasts, shouldn't it by default send
multicasts on every interface it actually uses?

I had expected ntpd would already do so.

Does someone know how this is handled by other protocols using multicast,
e.g. SLP or mDNS?


Martin
--
Martin Burnicki

Meinberg Funkuhren
Bad Pyrmont
Germany

Martin Burnicki wrote:
> Danny Mayer wrote:
>> John Vossler wrote:
> [...]
>>> My initial reaction to this issue was to determine if there was an OS
>>> (Solaris) directive to include the multicast for other interfaces in the
>>> group as reported by "netstat -g". Or a method to specify the multicast
>>> interfaces in the ntp.conf file (apparently not, or not easily).
>> No, and certainly it has not been implemented in NTP.
>
> If ntpd has been configured to send multicasts, shouldn't it by default send
> multicasts on every interface it actually uses?
>
> I had expected ntpd would already do so.
>

How could you do that? You have one packet that you are sending out. It
can only go out on exactly one interface. You can only choose which
interface to use, you cannot select more than one.

> Does someone know how this is handled by other protocols using multicast,
> e.g. SLP or mDNS?
>

They probably don't. Similarly DHCP.

Danny
>
> Martin

Danny Mayer wrote:

> Martin Burnicki wrote:
>> Danny Mayer wrote:
>>> John Vossler wrote:
>> [...]
>>>> My initial reaction to this issue was to determine if there was an OS
>>>> (Solaris) directive to include the multicast for other interfaces in
>>>> the
>>>> group as reported by "netstat -g". Or a method to specify the
>>>> multicast interfaces in the ntp.conf file (apparently not, or not
>>>> easily).
>>> No, and certainly it has not been implemented in NTP.
>>
>> If ntpd has been configured to send multicasts, shouldn't it by default
>> send multicasts on every interface it actually uses?
>>
>> I had expected ntpd would already do so.
>>
>
> How could you do that? You have one packet that you are sending out. It
> can only go out on exactly one interface. You can only choose which
> interface to use, you cannot select more than one.

I'd expect that either the kernel routed multicast packets to all interfaces
(isn't that what routers do with multicasts, contrarily to broadcasts?), or
the application would send an individual packet on each interface.

Anyway, if I have a server with several interfaces I'd expect those
multicasts to appear on every single subnet connected to one of those
interfaces.

Martin
--
Martin Burnicki

Meinberg Funkuhren
Bad Pyrmont
Germany

Martin Burnicki wrote:
> I'd expect that either the kernel routed multicast packets to all interfaces
> (isn't that what routers do with multicasts, contrarily to broadcasts?), or
> the application would send an individual packet on each interface.

Routers have to be told (by means of multicast protocols such as IGMP)
which interfaces to replicate multicast packets onto.

Jan

John Vossler wrote:
> Greetings,
>
> I have a new system running Solaris 10 set up as an NTP server. IT is
> synchronizing correctly but I cannot get it to multicast on any
> interface except the systems primary Ethernet interface, bge0. I need
> it to multicast on interfaces bge1-bge3 and ce0 - ce7.
>
> Client systems reside on the network segments on these other interfaces.
>
> Anyone have any suggestions on getting the server to multicast on these
> other interfaces?
>
> John

This issue has been discussed on the newsgroup around 2007-02-23. At
that time I taught ntpd to find the multicast default interface. This
solution is only suffcient on system with a single interface. To
quote the analysis from Bug #785
(https://support.ntp.org/bugs/show_bug.cgi?id=785):


As it turns out the IPv4 mcast implementation will copy the multicast
packets to any interface where the respective multicast groups have been
joined. For that to happen on a multihomed host (one with several
multicast capable interfaces) you need to run mrouted, zebra or some
other
multicast routing daemon.

So usually IPv4 MCAST will go via one interface. The kernel and/or
Multicast routing deamons then run the IGMP and other protocols to
find out which subscribers are connected on which interface and instruct
the kernel to do the respective packet replications.

On NetBSD (probably other BSDs too ans systems derived from that) my
setup is like this:

ntp.conf:

broadcast 224.0.1.1 ttl 5 autokey

I run mrouted.

and mrouted detects via IGMP my multicast clients. The multicast
routing table of the kernel is currently:


netstat -g

Virtual Interface Table
Vif Thresh Limit Local-Address Remote-Address Pkt_in Pkt_out
0 1 0 10.x.y.1 0 31
1 1 0 84.xxx.xxx.xxx 0 613

Multicast Forwarding Cache
Hash Origin Mcastgroup Traffic In-Vif Out-Vifs/Forw-ttl
38 84.xxx.xxx.xxx 224.0.1.1 613 1 0/1

Total no. of entries in cache: 1


This means for the Mcastgroup 224.0.1.1 (NTP.MCAST.NET) 613 packets
where sent via the interface Vif1 84.... These packets are replicated to
the interfaces Vif0 10... and Vif1 84.... for real output. You also
see that the interface Vif0 (10....) only sent 31 packets. This is
because I just started my notebook which is a multicast client.
As soon as ntpd on the notebook joined the Mcastgroup 224.0.1.1
pakets started flowing via the Vif0.

Solaris may or may not be needing a multicast routing daemon, but
I leave that research to someone else for now. But one thing to
note is that you may not see NTP Mcast traffic unless IGMP
messages are visible on the network segments (you can snoop for them).
So for debugging you need an ntpd on the interesting network segments
that listens for the multicast packets *before* you are likely to see
NTP multicast packets leaving the interface for that segment.
Usually Mcast packets are only send when necessary except for the
primary interface.

My current IGMP traffic looks like this (tcpdump -v -v -s 512 -i
igmp):
11:06:39.860323 IP (tos 0x0, ttl 1, id 0, offset 0, flags [none], proto
IGMP (2), length 28) server > ALL-SYSTEMS.MCAST.NET: igmp query v2
11:06:39.860404 IP (tos 0x0, ttl 1, id 0, offset 0, flags [none], proto
IGMP (2), length 32) server > DVMRP.MCAST.NET: igmp dvmrp Probe
genid 3783968839
11:06:45.498743 IP (tos 0x0, ttl 1, id 556, offset 0, flags [none],
proto IGMP (2), length 28) client > NTP.MCAST.NET: igmp v2 report
NTP.MCAST.NET
11:06:49.960402 IP (tos 0x0, ttl 1, id 0, offset 0, flags [none], proto
IGMP (2), length 32) server > DVMRP.MCAST.NET: igmp dvmrp Probe

Note the igmp v2 report line. Here the client documents interest in
NTP.MCAST.NET multicast packets which leads to forwarding the packets
to that network segment.

Hope this helps a bit.

Frank

Danny Mayer wrote:
> John Vossler wrote:
>> Greetings,
>>
>> I have a new system running Solaris 10 set up as an NTP server. IT is
>> synchronizing correctly but I cannot get it to multicast on any
>> interface except the systems primary Ethernet interface, bge0. I need
>> it to multicast on interfaces bge1-bge3 and ce0 - ce7.
>>
>> Client systems reside on the network segments on these other interfaces.
>>
>> Anyone have any suggestions on getting the server to multicast on these
>> other interfaces?
>>
>> John
>
> Are you talking about outgoing packets to a multicast address? I think
> that there may be some bugs in that area since the addresses are limited
> and choosing an outgoing address becomes an issue and it will only send
> it out on one address. I'm not sure if it's possible to set up a server
> (at least not easily) to send via different interfaces.
It is. But on a multihomed server the server must have multicast routing
functionality. Either this is already provided in the kernel or you
need to run an mcast-routing daemon like mrouted, zebra or others.
The ntpd setup is not different on multihomed hosts. Just the multicast
routing must be working.

>
> To do this correctly you need to be able to set up a different multicast
> address to use and associate it with a particular server address for
> each interface.
NO - see above.

> The only way that I can think of to accomplish this is
> to be able to specify a specific binding address in the config file. We
> would have to add that as an optional parameter to the broadcast directive.
>
not needed.
> Danny

Frank

Martin Burnicki wrote:
> Danny Mayer wrote:
>> John Vossler wrote:
> [...]
>>> My initial reaction to this issue was to determine if there was an OS
>>> (Solaris) directive to include the multicast for other interfaces in the
>>> group as reported by "netstat -g". Or a method to specify the multicast
>>> interfaces in the ntp.conf file (apparently not, or not easily).
>> No, and certainly it has not been implemented in NTP.
>
> If ntpd has been configured to send multicasts, shouldn't it by default send
> multicasts on every interface it actually uses?
No - replicating Mcast packets mustn't be the responsibility of the
application. That is the task of the multicast routing protocol software.

We really don't have to put all network software into ntpd :-).
>
> I had expected ntpd would already do so.
not needed.

>
> Does someone know how this is handled by other protocols using multicast,
> e.g. SLP or mDNS?
>
>
> Martin

Frank

Martin Burnicki wrote:
> Danny Mayer wrote:
>
>> Martin Burnicki wrote:
>>> Danny Mayer wrote:
>>>> John Vossler wrote:
>>> [...]
>>>>> My initial reaction to this issue was to determine if there was an OS
>>>>> (Solaris) directive to include the multicast for other interfaces in
>>>>> the
>>>>> group as reported by "netstat -g". Or a method to specify the
>>>>> multicast interfaces in the ntp.conf file (apparently not, or not
>>>>> easily).
>>>> No, and certainly it has not been implemented in NTP.
>>> If ntpd has been configured to send multicasts, shouldn't it by default
>>> send multicasts on every interface it actually uses?
>>>
>>> I had expected ntpd would already do so.
>>>
>> How could you do that? You have one packet that you are sending out. It
>> can only go out on exactly one interface. You can only choose which
>> interface to use, you cannot select more than one.
>
> I'd expect that either the kernel routed multicast packets to all interfaces
> (isn't that what routers do with multicasts, contrarily to broadcasts?),
Correct in both accounts. The kernel will do the replication if (!)
instructed to do so. Usually these instructions come from the multicast
routing software. That's why routers will do that and that's why your
multihomed system needs to become a multicast router.

or
> the application would send an individual packet on each interface.
>
> Anyway, if I have a server with several interfaces I'd expect those
> multicasts to appear on every single subnet connected to one of those
> interfaces.
That expectation is right, but the packets usually leave the interfaces
when the mcast routings software can determine that there is interest
for these packets. Basic interest is documented by client systems via
IGMP messages, but there there more that one multicast routing protocols
and that's why the routing policy is usually implemented outside the kernel
and the kernel only implements the forwarding mechanism.
>
> Martin

Frank

Frank Kardel wrote:
> Danny Mayer wrote:
>> John Vossler wrote:
>>> Greetings,
>>>
>>> I have a new system running Solaris 10 set up as an NTP server. IT
>>> is synchronizing correctly but I cannot get it to multicast on any
>>> interface except the systems primary Ethernet interface, bge0. I
>>> need it to multicast on interfaces bge1-bge3 and ce0 - ce7.
>>>
>>> Client systems reside on the network segments on these other interfaces.
>>>
>>> Anyone have any suggestions on getting the server to multicast on
>>> these other interfaces?
>>>
>>> John
>>
>> Are you talking about outgoing packets to a multicast address? I think
>> that there may be some bugs in that area since the addresses are
>> limited and choosing an outgoing address becomes an issue and it will
>> only send it out on one address. I'm not sure if it's possible to set
>> up a server (at least not easily) to send via different interfaces.
> It is. But on a multihomed server the server must have multicast routing
> functionality. Either this is already provided in the kernel or you
> need to run an mcast-routing daemon like mrouted, zebra or others.
> The ntpd setup is not different on multihomed hosts. Just the multicast
> routing must be working.
>

Even if that's true, we don't know apriori whether or not that is the
*intent* of the sysop. Most times you are likely to have a server with
two NICs but one is used as an outbound interface to the external
servers and the other NIC is used to multicast packets to the LAN. NTPD
has no way of knowing that. I assume that something like mrouted can be
configured in some way to know how to send those packets. Note that such
a configuration is likely to introduce additional delays and jitter to
the outgoing multicast packet.

>>
>> To do this correctly you need to be able to set up a different
>> multicast address to use and associate it with a particular server
>> address for each interface.
> NO - see above.

Yes, it turns out you can do this from within ntpd but you have to
program this.

Danny
>
>> The only way that I can think of to accomplish this is to be able to
>> specify a specific binding address in the config file. We would have
>> to add that as an optional parameter to the broadcast directive.
>>
> not needed.
>> Danny
>
> Frank
>

Danny Mayer wrote:
> Frank Kardel wrote:
>> Danny Mayer wrote:
>>> John Vossler wrote:
>>>> Greetings,
>>>>
>>>> I have a new system running Solaris 10 set up as an NTP server. IT
>>>> is synchronizing correctly but I cannot get it to multicast on any
>>>> interface except the systems primary Ethernet interface, bge0. I
>>>> need it to multicast on interfaces bge1-bge3 and ce0 - ce7.
>>>>
>>>> Client systems reside on the network segments on these other
>>>> interfaces.
>>>>
>>>> Anyone have any suggestions on getting the server to multicast on
>>>> these other interfaces?
>>>>
>>>> John
>>>
>>> Are you talking about outgoing packets to a multicast address? I
>>> think that there may be some bugs in that area since the addresses
>>> are limited and choosing an outgoing address becomes an issue and it
>>> will only send it out on one address. I'm not sure if it's possible
>>> to set up a server (at least not easily) to send via different
>>> interfaces.
>> It is. But on a multihomed server the server must have multicast routing
>> functionality. Either this is already provided in the kernel or you
>> need to run an mcast-routing daemon like mrouted, zebra or others.
>> The ntpd setup is not different on multihomed hosts. Just the multicast
>> routing must be working.
>>
>
> Even if that's true, we don't know apriori whether or not that is the
> *intent* of the sysop.
> Most times you are likely to have a server with two NICs but one is
> used as an outbound interface to the external servers and the other
> NIC is used to multicast packets to the LAN. NTPD has no way of
> knowing that.
It doesn't need to know that. Basically multicast configuration is sort
of configured backwards. The clients subscribe and the (multicast
routers) servers will supply the data where it is subscribed to (in
normal configuration),
The whole point about multicast is to be able to provide data to a large
interested group and at the same time limit the distribution to the only
necessary distribution tree.
The only issue we might have (again) is what source address would we
like to see in these multicast and would we be willing to see all
multicasts from the server for all/some local server interfaces?
> I assume that something like mrouted can be configured in some way to
> know how to send those packets. Note that such a configuration is
> likely to introduce additional delays and jitter to the outgoing
> multicast packet.
>
>>>
>>> To do this correctly you need to be able to set up a different
>>> multicast address to use and associate it with a particular server
>>> address for each interface.
>> NO - see above.
>
> Yes, it turns out you can do this from within ntpd but you have to
> program this.
You can do many things - you can even use the same multicast address
outbound by sending over different sockets. On a multicast router this
would lead to different source addresses
seen by the multicast clients. On a non multicast router each segment
would only see the multicast packets from the interface serving them.
Usually this effort is not needed when (mcast)
routing between al segments is possible. Things get tedious when ntpd
runs on a platform that does not (by choice / misconfiguration) perform
the routing. Then requirements for
supporting split views of the ntp service come up.

Generalisation:
The issue of local server addresses basically boils down to the ability
the configure the / address pair for associations - no
matter whether they are peers/servers/broadcasts.
This will also solve all current ambiguities with the proposed query-on
directive. We could have several ways to specify the local address:

* - fully automatic - usually best and works for almost all environments
as it matches routing
(interface) - pick one of the interface addresses within the protocol
family - this one is already semantically difficult - single address
interfaces would work best here, probably that form
should generate a group of associations the match the addresses of the
given interface.
IP address - a single valid local address - this is the most precise
definition but is also most likely to violate routing information when
used with a specific destination address.

The point is that associations would actually need to be configured on a
src/dst address basis to cover all these exotic network setups where
somebody put an ntp server onto a central network component. That way he
runs into all configuration issues of a (mcast-)router and it is not
even sure that this network device would be an adaequate platform for
ntpd performancewise.

While I have a strategy in my mind to cover all these cases (see above -
association local address specifications) I am not sure whether we want
to go through that trouble.

Frank

Jan Ceuleers wrote:
> Martin Burnicki wrote:
>> I'd expect that either the kernel routed multicast packets to all interfaces
>> (isn't that what routers do with multicasts, contrarily to broadcasts?), or
>> the application would send an individual packet on each interface.
>
> Routers have to be told (by means of multicast protocols such as IGMP)
> which interfaces to replicate multicast packets onto.
>

That assumes that there's a router involved. If it's just to the local
LAN then you may not have a router able to route to the other
interfaces. In this case it almost certainly have to be done on the
local server sending the packets.

Danny

Frank,

thanks for that crash course on the details of multicasting which makes
things much clearer.

Martin
--
Martin Burnicki

Meinberg Funkuhren
Bad Pyrmont
Germany

Danny Mayer wrote:
> Jan Ceuleers wrote:
>> Martin Burnicki wrote:
>>> I'd expect that either the kernel routed multicast packets to all interfaces
>>> (isn't that what routers do with multicasts, contrarily to broadcasts?), or
>>> the application would send an individual packet on each interface.
>> Routers have to be told (by means of multicast protocols such as IGMP)
>> which interfaces to replicate multicast packets onto.
>>
>
> That assumes that there's a router involved. If it's just to the local
> LAN then you may not have a router able to route to the other
> interfaces. In this case it almost certainly have to be done on the
> local server sending the packets.
>
> Danny
Well, when you install/enable multicast routing software on the local
machine this machine becomes a multicast router (kernel may have to
be compiled with the right flags on some systems). Thus the local
machine will be NTP server and multicast router.As I said you need
the multicast routing functionality in order to be able to find the
interfaces where you need to replicate to packets to.

Frank