vxworks6.6 : running multiple AMP instances on multicore chips - VxWorks

This is a discussion on vxworks6.6 : running multiple AMP instances on multicore chips - VxWorks ; Hi In vxWorks6.6, there is a restriction that vxworks bootloader should not have SMP enabled. That means, vxworks bootloader is capable of running on only single processor. So, how to load AMP images (multiple instances of vxworks) on multiple CPUs? ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: vxworks6.6 : running multiple AMP instances on multicore chips

  1. vxworks6.6 : running multiple AMP instances on multicore chips

    Hi

    In vxWorks6.6, there is a restriction that vxworks bootloader should
    not have SMP enabled. That means, vxworks bootloader is capable of
    running on only single processor. So, how to load AMP images
    (multiple instances of vxworks) on multiple CPUs? (in a multi core
    chip)

    I have MIPS based mutli core chip, now I can run one instance of
    vxworks SMP or multiple instances of vxworks AMP. In later case, what
    are the best ways to design AMP instances? One of the wind river docs
    sugests that one instance use the kseg0 and remaining other instances
    can be run from the kseg1 or kuseg in a mapped kernel mode. Can
    anybody explain this in detail? What are the implications in
    designing other instances to be in kseg1 or kuseg? Is there any
    sugested doc from wind river?

    Currently vxworks6.6 supports cavium and Raza XLR multicore
    processors. If I build the image in AMP mode, I see only one image
    get builded. How can I run this on multiple cpus?

    thanks,
    MB.

  2. Re: vxworks6.6 : running multiple AMP instances on multicore chips

    On Mar 31, 10:26 am, MB wrote:
    > Hi
    >
    > In vxWorks6.6, there is a restriction that vxworks bootloader should
    > not have SMP enabled. That means, vxworks bootloader is capable of
    > running on only single processor. So, how to load AMP images
    > (multiple instances of vxworks) on multiple CPUs? (in a multi core
    > chip)
    >
    > I have MIPS based mutli core chip, now I can run one instance of
    > vxworks SMP or multiple instances of vxworks AMP. In later case, what
    > are the best ways to design AMP instances? One of the wind river docs
    > sugests that one instance use the kseg0 and remaining other instances
    > can be run from the kseg1 or kuseg in a mapped kernel mode. Can
    > anybody explain this in detail? What are the implications in
    > designing other instances to be in kseg1 or kuseg? Is there any
    > sugested doc from wind river?
    >
    > Currently vxworks6.6 supports cavium and Raza XLR multicore
    > processors. If I build the image in AMP mode, I see only one image
    > get builded. How can I run this on multiple cpus?
    >
    > thanks,
    > MB.



    As far as I know, the Raza Micro BSP as shipped doesn't support an AMP
    configuration out of the box, only SMP. If you look at the
    '20bsp.cdf' file that comes with it, the 'Bsp' specification only
    lists SMP under MT_OPTIONS.

    You could attempt to create your own BSP for this, but there are a
    couple off issues. First, you need to have different images linked for
    different addresses: there's no trick available for remapping memory
    so that each core sees the same logical address space. Second, only
    the first image can include the network driver, since there is only
    one instance of the interface used for all 4 RGMII network ports: if
    you include the driver in more than one image, both will attempt to
    manipulate the same device and collide with each other. (I believe the
    same issue exists with the serial ports and PCI bus.) As for getting
    the other cores bootstrapped, ultimately, I think what you have to do
    use the image running on CPU 0 to load subsequent images for the other
    cores.

    That said, with the Raza Micro and Cavium designs, the best way to
    take advantage of the hardware is not to simply run multiple instances
    of a full blown OS on each core (or even one instance in SMP): what
    you're supposed to do is run dedicated standalone threads on each CPU
    which pass work back and forth to each other using the hardware work
    queue. You can use VxWorks (or some other OS) on one core to manage
    everything, but putting the same OS on every core is probably not the
    best approach.

    -Bill

  3. Re: vxworks6.6 : running multiple AMP instances on multicore chips

    On Apr 1, 4:34*pm, noiset...@gmail.com wrote:
    > On Mar 31, 10:26 am, MB wrote:
    >
    >
    >
    >
    >
    > > Hi

    >
    > > In vxWorks6.6, there is a restriction that vxworks bootloader should
    > > not have SMP *enabled. *That means, vxworks bootloader is capable of
    > > running on only single processor. * So, how to load AMP images
    > > (multiple instances of vxworks) on multiple CPUs? (in a multi core
    > > chip)

    >
    > > I have MIPS based mutli core chip, now I can run one instance of
    > > vxworks SMP or multiple instances of vxworks AMP. In later case, *what
    > > are the best ways to design AMP instances? *One of the wind river docs
    > > sugests that one instance use the kseg0 and remaining other instances
    > > can be run from the kseg1 or kuseg in a mapped kernel mode. *Can
    > > anybody explain this in detail? *What are the implications in
    > > designing other instances to be in kseg1 or kuseg? *Is there any
    > > sugested doc from wind river?

    >
    > > Currently vxworks6.6 supports cavium and Raza XLR multicore
    > > processors. If I build the image in AMP mode, *I see only one image
    > > get builded. How can I run this on multiple cpus?

    >
    > > thanks,
    > > MB.

    >
    > As far as I know, the Raza Micro BSP as shipped doesn't support an AMP
    > configuration out of the box, only SMP. *If you look at the
    > '20bsp.cdf' file that comes with it, the 'Bsp' specification only
    > lists SMP under MT_OPTIONS.
    >
    > You could attempt to create your own BSP for this, but there are a
    > couple off issues. First, you need to have different images linked for
    > different addresses: there's no trick available for remapping memory
    > so that each core sees the same logical address space. Second, only
    > the first image can include the network driver, since there is only
    > one instance of the interface used for all 4 RGMII network ports: if
    > you include the driver in more than one image, both will attempt to
    > manipulate the same device and collide with each other. (I believe the
    > same issue exists with the serial ports and PCI bus.) As for getting
    > the other cores bootstrapped, ultimately, I think what you have to do
    > use the image running on CPU 0 to load subsequent images for the other
    > cores.
    >
    > That said, with the Raza Micro and Cavium designs, the best way to
    > take advantage of the hardware is not to simply run multiple instances
    > of a full blown OS on each core (or even one instance in SMP): what
    > you're supposed to do is run dedicated standalone threads on each CPU
    > which pass work back and forth to each other using the hardware work
    > queue. You can use VxWorks (or some other OS) on one core to manage
    > everything, but putting the same OS on every core is probably not the
    > best approach.
    >
    > -Bill- Hide quoted text -
    >
    > - Show quoted text -


    Madhu,
    Bill has given very information here. Let me add what Bill has already
    stated above.

    In an AMP configuration, you can NOT afford to put "same hardware
    resources" available to all CPUs simultaneousely. They have to be
    dedicated to any one of the CPUs. They can however share information
    by means of "shared memory".
    ( That is what Bill pointed out). I guess he can still run the way he
    has originally envisioned, provided, there is no multiple driver
    copies across the CPUs for the same set hardware devices i.e. for
    example, if there is a NIC interface, the driver code for this NIC
    device should run only on one CPU and no other CPU should access this
    device.

    That said, in an SMP configuration, perhaps you may examine the CPU
    affinity API. Using this API, you can chose to execute a particular
    task on a particular CPU or a particular set of CPUs. However, such an
    assignment of CPUs for those tasks should be done after a carefull
    analysis and / or profiling of over all execution of all the tasks.
    Without such analysis, the overall performance may be in fact very
    poor ( That may be surprising, but it can be true).
    -Venkat Dyavana




  4. Re: vxworks6.6 : running multiple AMP instances on multicore chips

    Hi BIll, Venkat Dyavana,

    Thanks for your opinions. I agree that not all cpus can control all
    the resources. We can design that resources can be programmed and
    controled with the master OS, which runs in the CPU0. Remaining all
    cpus can either run "native other OS" or slim down version of vxworks
    itself. In later case, slim down means, they still use the hw
    resources provided, devices gives access to these cpus. But those
    devices will be controlled by CPU0 (os run by cpu0)only.

    Master OS, which will run in cpu0 will be conventional vxworks which
    will run in kseg0 or in kseg1 if MMU enabled. Now, to design those
    slim down version of vxworks, which is best way of doing? Make it
    still kseg1 based with memory mapped or kuseg based? Assume, master
    OS will later launch other os on other cpus. Does any one have these
    models or do you think any problems in this design regarding handling
    MMU or RTP related in non master OS?

    thanks,
    MB.


    On Apr 2, 3:32*am, Dyava...@gmail.com wrote:
    > On Apr 1, 4:34*pm, noiset...@gmail.com wrote:
    >
    >
    >
    >
    >
    > > On Mar 31, 10:26 am, MB wrote:

    >
    > > > Hi

    >
    > > > In vxWorks6.6, there is a restriction that vxworks bootloader should
    > > > not have SMP *enabled. *That means, vxworks bootloader is capable of
    > > > running on only single processor. * So, how to load AMP images
    > > > (multiple instances of vxworks) on multiple CPUs? (in a multi core
    > > > chip)

    >
    > > > I have MIPS based mutli core chip, now I can run one instance of
    > > > vxworks SMP or multiple instances of vxworks AMP. In later case, *what
    > > > are the best ways to design AMP instances? *One of the wind river docs
    > > > sugests that one instance use the kseg0 and remaining other instances
    > > > can be run from the kseg1 or kuseg in a mapped kernel mode. *Can
    > > > anybody explain this in detail? *What are the implications in
    > > > designing other instances to be in kseg1 or kuseg? *Is there any
    > > > sugested doc from wind river?

    >
    > > > Currently vxworks6.6 supports cavium and Raza XLR multicore
    > > > processors. If I build the image in AMP mode, *I see only one image
    > > > get builded. How can I run this on multiple cpus?

    >
    > > > thanks,
    > > > MB.

    >
    > > As far as I know, the Raza Micro BSP as shipped doesn't support an AMP
    > > configuration out of the box, only SMP. *If you look at the
    > > '20bsp.cdf' file that comes with it, the 'Bsp' specification only
    > > lists SMP under MT_OPTIONS.

    >
    > > You could attempt to create your own BSP for this, but there are a
    > > couple off issues. First, you need to have different images linked for
    > > different addresses: there's no trick available for remapping memory
    > > so that each core sees the same logical address space. Second, only
    > > the first image can include the network driver, since there is only
    > > one instance of the interface used for all 4 RGMII network ports: if
    > > you include the driver in more than one image, both will attempt to
    > > manipulate the same device and collide with each other. (I believe the
    > > same issue exists with the serial ports and PCI bus.) As for getting
    > > the other cores bootstrapped, ultimately, I think what you have to do
    > > use the image running on CPU 0 to load subsequent images for the other
    > > cores.

    >
    > > That said, with the Raza Micro and Cavium designs, the best way to
    > > take advantage of the hardware is not to simply run multiple instances
    > > of a full blown OS on each core (or even one instance in SMP): what
    > > you're supposed to do is run dedicated standalone threads on each CPU
    > > which pass work back and forth to each other using the hardware work
    > > queue. You can use VxWorks (or some other OS) on one core to manage
    > > everything, but putting the same OS on every core is probably not the
    > > best approach.

    >
    > > -Bill- Hide quoted text -

    >
    > > - Show quoted text -

    >
    > Madhu,
    > Bill has given very information here. Let me add what Bill has already
    > stated above.
    >
    > In an AMP configuration, you can NOT afford to put "same hardware
    > resources" available to all CPUs simultaneousely. They have to be
    > dedicated to any one of the CPUs. They can however share information
    > by means of "shared memory".
    > ( That is what Bill pointed out). I guess he can still run the way he
    > has originally envisioned, provided, there is no multiple driver
    > copies across the CPUs for the same set hardware devices i.e. for
    > example, if there is a NIC interface, the driver code for this NIC
    > device should run only on one CPU and no other CPU should access this
    > device.
    >
    > That said, in an SMP configuration, perhaps you may examine the CPU
    > affinity API. Using this API, you can chose to execute a particular
    > task on a particular CPU or a particular set of CPUs. However, such an
    > assignment of CPUs for those tasks should be done after a *carefull
    > analysis and / or profiling of *over all execution of all the tasks.
    > Without such analysis, the overall performance may be in fact very
    > poor ( That may be surprising, but it can be true).
    > -Venkat Dyavana- Hide quoted text -
    >
    > - Show quoted text -



+ Reply to Thread