Use of barriers in pvclock ABI - Kernel

This is a discussion on Use of barriers in pvclock ABI - Kernel ; In Xen, we guarantee that the pv clock record will only ever be updated by the current cpu, so there will never be any cross-cpu barrier or synchronization issues to consider, nor any chance a vcpu will see its own ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: Use of barriers in pvclock ABI

  1. Use of barriers in pvclock ABI

    In Xen, we guarantee that the pv clock record will only ever be updated
    by the current cpu, so there will never be any cross-cpu barrier or
    synchronization issues to consider, nor any chance a vcpu will see its
    own clock record in a partially updated state.

    I notice the current kvm implementation is the same.

    However, the pvclock_clocksource_read() implementation is
    over-engineered, because it checks for an odd version and uses very
    strong rmb() barriers (which generates either an "lfence" or "lock add
    $0, (%esp)").

    If we're happy to guarantee as an ABI issue that the record will never
    be updated cross-cpu, then we can make the barriers simply barrier() and
    just check for (src->version != dst->version).

    Is that OK with you, or do you want to leave open the possibility of
    doing cross-cpu time updates?

    J
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  2. Re: Use of barriers in pvclock ABI

    Hi,

    > However, the pvclock_clocksource_read() implementation is
    > over-engineered, because it checks for an odd version and uses very
    > strong rmb() barriers (which generates either an "lfence" or "lock add
    > $0, (%esp)").
    >
    > If we're happy to guarantee as an ABI issue that the record will never
    > be updated cross-cpu, then we can make the barriers simply barrier() and
    > just check for (src->version != dst->version).
    >
    > Is that OK with you, or do you want to leave open the possibility of
    > doing cross-cpu time updates?


    Due to the TSC being involved here I don't expect cross-cpu time updates
    will ever happen. IMHO it is fine to change that.

    cheers,
    Gerd

    --
    http://kraxel.fedorapeople.org/xenner/
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  3. Re: Use of barriers in pvclock ABI

    Gerd Hoffmann wrote:
    > Hi,
    >
    >
    >> However, the pvclock_clocksource_read() implementation is
    >> over-engineered, because it checks for an odd version and uses very
    >> strong rmb() barriers (which generates either an "lfence" or "lock add
    >> $0, (%esp)").
    >>
    >> If we're happy to guarantee as an ABI issue that the record will never
    >> be updated cross-cpu, then we can make the barriers simply barrier() and
    >> just check for (src->version != dst->version).
    >>
    >> Is that OK with you, or do you want to leave open the possibility of
    >> doing cross-cpu time updates?
    >>

    >
    > Due to the TSC being involved here I don't expect cross-cpu time updates
    > will ever happen. IMHO it is fine to change that.
    >
    >


    I agree. And if we ever feel the need, we can allocate a feature bit
    for it.

    --
    error compiling committee.c: too many arguments to function

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  4. Re: Use of barriers in pvclock ABI

    On Mon, Aug 11, 2008 at 4:08 AM, Gerd Hoffmann wrote:
    > Hi,
    >
    >> However, the pvclock_clocksource_read() implementation is
    >> over-engineered, because it checks for an odd version and uses very
    >> strong rmb() barriers (which generates either an "lfence" or "lock add
    >> $0, (%esp)").
    >>
    >> If we're happy to guarantee as an ABI issue that the record will never
    >> be updated cross-cpu, then we can make the barriers simply barrier() and
    >> just check for (src->version != dst->version).
    >>
    >> Is that OK with you, or do you want to leave open the possibility of
    >> doing cross-cpu time updates?

    >
    > Due to the TSC being involved here I don't expect cross-cpu time updates
    > will ever happen. IMHO it is fine to change that.


    Okay for guest vcpu, but what about physical cpus?

    IIRC, the checks are there, and so strict, to account for the
    possiblity of the vcpu to be migrated to another cpu in the middle of
    the
    clock reading.
    >
    > cheers,
    > Gerd
    >
    > --
    > http://kraxel.fedorapeople.org/xenner/
    >




    --
    Glauber Costa.
    "Free as in Freedom"
    http://glommer.net

    "The less confident you are, the more serious you have to act."
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  5. Re: Use of barriers in pvclock ABI

    Glauber Costa wrote:
    > On Mon, Aug 11, 2008 at 4:08 AM, Gerd Hoffmann wrote:
    >
    >> Hi,
    >>
    >>
    >>> However, the pvclock_clocksource_read() implementation is
    >>> over-engineered, because it checks for an odd version and uses very
    >>> strong rmb() barriers (which generates either an "lfence" or "lock add
    >>> $0, (%esp)").
    >>>
    >>> If we're happy to guarantee as an ABI issue that the record will never
    >>> be updated cross-cpu, then we can make the barriers simply barrier() and
    >>> just check for (src->version != dst->version).
    >>>
    >>> Is that OK with you, or do you want to leave open the possibility of
    >>> doing cross-cpu time updates?
    >>>

    >> Due to the TSC being involved here I don't expect cross-cpu time updates
    >> will ever happen. IMHO it is fine to change that.
    >>

    >
    > Okay for guest vcpu, but what about physical cpus?
    >
    >
    > IIRC, the checks are there, and so strict, to account for the
    > possiblity of the vcpu to be migrated to another cpu in the middle of
    > the
    >



    Migration implies an smp barrier (but not a compiler barrier, of
    course). As to the the version number oddness check, I don't see how it
    can be necessary.


    --
    error compiling committee.c: too many arguments to function

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  6. Re: Use of barriers in pvclock ABI

    Glauber Costa wrote:
    > On Mon, Aug 11, 2008 at 4:08 AM, Gerd Hoffmann wrote:
    >> Due to the TSC being involved here I don't expect cross-cpu time updates
    >> will ever happen. IMHO it is fine to change that.

    >
    > Okay for guest vcpu, but what about physical cpus?
    >
    > IIRC, the checks are there, and so strict, to account for the
    > possiblity of the vcpu to be migrated to another cpu in the middle of
    > the
    > clock reading.


    This is about the check in pvclock_get_time_values() that it got a
    consistent snapshot. Dropping that is fine.

    pvclock_clocksource_read() will still notice when being migrated to
    another pcpu in the middle of the clock reading.

    cheers,
    Gerd

    --
    http://kraxel.fedorapeople.org/xenner/
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  7. Re: Use of barriers in pvclock ABI

    Glauber Costa wrote:
    > Okay for guest vcpu, but what about physical cpus?
    >
    > IIRC, the checks are there, and so strict, to account for the
    > possiblity of the vcpu to be migrated to another cpu in the middle of
    > the
    > clock reading.
    >


    That's fine. As part of rescheduling a vcpu on a new pcpu, the clock
    record will be updated with the new cpu's parameters, but that update
    will be complete by the time the vcpu gets rescheduled. The version
    check and loop still needs to be there, but it will never see an
    inconsistent (partially updated) clock record.

    J
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

+ Reply to Thread