Can VME Windows and PCI Space share memory addresses? - VxWorks

This is a discussion on Can VME Windows and PCI Space share memory addresses? - VxWorks ; Hello, I'm trying to migrate to a Synergy MANTA DX3 dual 7457 CPU SBC. I have a DSS networks 5164 4-port GB Ethernet card I'm trying to get to work on the board, under vxWorks 5.5.1, and it just won't ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: Can VME Windows and PCI Space share memory addresses?

  1. Can VME Windows and PCI Space share memory addresses?

    Hello,

    I'm trying to migrate to a Synergy MANTA DX3 dual 7457 CPU SBC. I have
    a DSS networks 5164 4-port GB Ethernet card I'm trying to get to work
    on the board, under vxWorks 5.5.1, and it just won't work, can't get it
    loaded into the MUX without locking the board up in "funny" ways. And
    it depends on which port I try to load into the MUX as well, which
    seems strange.

    In the past few days I had an unrelated VME Window problem (which I
    think I now understand and have solved), and it just dawned on me, that
    I don't think that you can have the board set up with PCI space, and
    VME window(s) at the same memory addresses, right? This would mean the
    VME windows and PCI would conflict I think, is that true?
    I don't think I can even have the two PCI spaces overlapping, or that
    would be extremely bad as well, right?

    For example, Synergy sets up 3 VME windows in the BSP by default, and I
    remember at least 2 of them being at 0x90M and 0xa0M (don't have the
    board in frot of me).
    This board also has 2 PCI spaces, PCI0 and PCI1. PCI 0 was set up at
    0x80M with some size (either 256, 512, or 768 MB), and PCI1 was set up
    at 0xb0M with 512MB size.
    This appears that maybe the PCI spaces are even conflicting with
    themselves. But on top of that, the VME window at 0xb0M would
    definitely conflict with the DSS card in that specific slot. This makes
    sense to me as why I can't get the card working, and it does "weird"
    things.

    On top of all that, on the board pciConfigTopoShow won't show anything
    on anothe bus but the PCI0 bus, which might be a sign of this
    conflicting.

    Could anyone confirm that all these memory addresses MUST be different
    and not overlap? If so that would give me some confidence in dealing
    with Synergy.They aren't that helpful lately.


  2. Re: Can VME Windows and PCI Space share memory addresses?

    Hi:

    I think what might be confusing is that the VME is probably bridge via
    a Universe chip onto the PCI space. So, you should see VME addresses
    somewhere within your whole range of pci addresses. This is the most
    common way of doing things.

    Also, don't confuse VME addresses (ie those on the VME bus) with the
    CPUs address space, of which the PCI belongs. You have to do all the
    math to convert between them, or use the bsp functions
    sysBusToLocalAdrs (to trans vme to cpu), and sysLocalToBusAdrs to go
    the other way.

    I don't follow how your DSS card is installed. I'm guessing it is on a
    PMC site and not a VME card?

    The only problem I can recall causing your type of issues is that the
    TOTAL address range on the local bus allocated to PCI is exceeded due
    to expanding the VME address range such that all the PCI devices and
    vme address no longer fit. If that's really happening, then there
    should be some BSP #define that can increase the size of the total pci
    window that the can be used to accomodate all your pci devices, vme
    windows, etc.

    sysPciShow and sysVmeShow should help if they are in your BSP.

    Good luck,
    lc
    Jammer wrote:
    > Hello,
    >
    > I'm trying to migrate to a Synergy MANTA DX3 dual 7457 CPU SBC. I have
    > a DSS networks 5164 4-port GB Ethernet card I'm trying to get to work
    > on the board, under vxWorks 5.5.1, and it just won't work, can't get it
    > loaded into the MUX without locking the board up in "funny" ways. And
    > it depends on which port I try to load into the MUX as well, which
    > seems strange.
    >
    > In the past few days I had an unrelated VME Window problem (which I
    > think I now understand and have solved), and it just dawned on me, that
    > I don't think that you can have the board set up with PCI space, and
    > VME window(s) at the same memory addresses, right? This would mean the
    > VME windows and PCI would conflict I think, is that true?
    > I don't think I can even have the two PCI spaces overlapping, or that
    > would be extremely bad as well, right?
    >
    > For example, Synergy sets up 3 VME windows in the BSP by default, and I
    > remember at least 2 of them being at 0x90M and 0xa0M (don't have the
    > board in frot of me).
    > This board also has 2 PCI spaces, PCI0 and PCI1. PCI 0 was set up at
    > 0x80M with some size (either 256, 512, or 768 MB), and PCI1 was set up
    > at 0xb0M with 512MB size.
    > This appears that maybe the PCI spaces are even conflicting with
    > themselves. But on top of that, the VME window at 0xb0M would
    > definitely conflict with the DSS card in that specific slot. This makes
    > sense to me as why I can't get the card working, and it does "weird"
    > things.
    >
    > On top of all that, on the board pciConfigTopoShow won't show anything
    > on anothe bus but the PCI0 bus, which might be a sign of this
    > conflicting.
    >
    > Could anyone confirm that all these memory addresses MUST be different
    > and not overlap? If so that would give me some confidence in dealing
    > with Synergy.They aren't that helpful lately.



  3. Re: Can VME Windows and PCI Space share memory addresses?

    Thanks very much for this info.
    Since I posted this, I've talked to the Vendor and slightly more
    understand what's going on. There's a "tempe" Tundra chip on the PCI #0
    bus which is the VME bridge.
    The DSS card is a PCI PMC card in one of the PMC slots on the SBC (it
    has 2).

    Today and yesterday I did some testing, and it's sort of strange that I
    can get the card mostly working in one of the PCI slots, but not in the
    other one. I also have a ScramNet GT PMC card, and it also works
    "funny" in this same slot, but works ok in the other slot.

    What I'm confused about it that I have 3 "Master" VME windows
    configured within the PCI #0 space when I use sysMasterPortsShow(). I'm
    not sure if that's correct or not. I can see how there needs to be
    access to the tundra chip itself within this PCI #0 space, but I guess
    I'm confused why the Master Windows need to be there, maybe becuase
    they're really part of the Tundra chip? Shows you how little I know
    about VME and the access to it.

    I am starting to make the distinction and understand PowerPC memory
    access vs. the VME bus addresses. I'm only really concerned about the
    addresses within my programming environment conflicting, meaning, what
    the PowerPC sees (e.g., pointer values)

    The vendor is giving me a very hard time that we're trying to integrate
    a "3rd party card" with thier board/BSP, where I say that any PCI card
    plugged into the PMC slot should be "seen" by pciConfigTopoShow, which
    is NOT the case for anything plugged into this specific PMC site.

    Thanks again.

    LarryC wrote:
    > Hi:
    >
    > I think what might be confusing is that the VME is probably bridge via
    > a Universe chip onto the PCI space. So, you should see VME addresses
    > somewhere within your whole range of pci addresses. This is the most
    > common way of doing things.
    >
    > Also, don't confuse VME addresses (ie those on the VME bus) with the
    > CPUs address space, of which the PCI belongs. You have to do all the
    > math to convert between them, or use the bsp functions
    > sysBusToLocalAdrs (to trans vme to cpu), and sysLocalToBusAdrs to go
    > the other way.
    >
    > I don't follow how your DSS card is installed. I'm guessing it is on a
    > PMC site and not a VME card?
    >
    > The only problem I can recall causing your type of issues is that the
    > TOTAL address range on the local bus allocated to PCI is exceeded due
    > to expanding the VME address range such that all the PCI devices and
    > vme address no longer fit. If that's really happening, then there
    > should be some BSP #define that can increase the size of the total pci
    > window that the can be used to accomodate all your pci devices, vme
    > windows, etc.
    >
    > sysPciShow and sysVmeShow should help if they are in your BSP.
    >
    > Good luck,
    > lc
    > Jammer wrote:
    > > Hello,
    > >
    > > I'm trying to migrate to a Synergy MANTA DX3 dual 7457 CPU SBC. I have
    > > a DSS networks 5164 4-port GB Ethernet card I'm trying to get to work
    > > on the board, under vxWorks 5.5.1, and it just won't work, can't get it
    > > loaded into the MUX without locking the board up in "funny" ways. And
    > > it depends on which port I try to load into the MUX as well, which
    > > seems strange.
    > >
    > > In the past few days I had an unrelated VME Window problem (which I
    > > think I now understand and have solved), and it just dawned on me, that
    > > I don't think that you can have the board set up with PCI space, and
    > > VME window(s) at the same memory addresses, right? This would mean the
    > > VME windows and PCI would conflict I think, is that true?
    > > I don't think I can even have the two PCI spaces overlapping, or that
    > > would be extremely bad as well, right?
    > >
    > > For example, Synergy sets up 3 VME windows in the BSP by default, and I
    > > remember at least 2 of them being at 0x90M and 0xa0M (don't have the
    > > board in frot of me).
    > > This board also has 2 PCI spaces, PCI0 and PCI1. PCI 0 was set up at
    > > 0x80M with some size (either 256, 512, or 768 MB), and PCI1 was set up
    > > at 0xb0M with 512MB size.
    > > This appears that maybe the PCI spaces are even conflicting with
    > > themselves. But on top of that, the VME window at 0xb0M would
    > > definitely conflict with the DSS card in that specific slot. This makes
    > > sense to me as why I can't get the card working, and it does "weird"
    > > things.
    > >
    > > On top of all that, on the board pciConfigTopoShow won't show anything
    > > on anothe bus but the PCI0 bus, which might be a sign of this
    > > conflicting.
    > >
    > > Could anyone confirm that all these memory addresses MUST be different
    > > and not overlap? If so that would give me some confidence in dealing
    > > with Synergy.They aren't that helpful lately.



  4. Re: Can VME Windows and PCI Space share memory addresses?

    I'm guessing you have more than one PCI bus on your board. Each PMC
    slot is probably on a different bus, so each will require a separate
    call to see the devices.

    On my system, there are two buses, and they are 0, and 0x20.
    value = 0 = 0x0
    xxx[cpu0]->pciDeviceShow
    Scanning function 0 of each PCI device on bus 0
    Using configuration mechanism 0
    bus device function vendorID deviceID class
    00000000 00000000 00000000 000011ab 00006480 00058000
    00000000 00000004 00000000 000011b5 00000014 00068000
    value = 0 = 0x0
    xxx[cpu0]->pciDeviceShow 0x20
    Scanning function 0 of each PCI device on bus 32
    Using configuration mechanism 0
    bus device function vendorID deviceID class
    00000020 00000000 00000000 000011ab 00006480 00058000
    00000020 00000002 00000000 000010b5 00006520 00060400
    00000020 00000004 00000000 00001013 00001113 00060700
    value = 0 = 0x0
    Orion DY4[cpu0]->


    The trick is that pciConfigTopoShow is hardcoded to do only bus0, so
    you have to force it manually to do the other bus. Take a look at the
    source.

    Here's the way I do it...
    value = 0 = 0x0
    Orion DY4[cpu0]->pciConfigForeachFunc 0x20,0xff,pciConfigFuncShow,0

    [32,0,0] type=MEM_CNTLR
    status=0x22b0 ( CAP 66MHZ FBTB DEVSEL=1 MSTR_ABORT_RCV )
    command=0x01d7 ( IO_ENABLE MEM_ENABLE MASTER_ENABLE WI_ENABLE
    PERR_ENABL
    E WC_ENABLE SERR_ENABLE )
    bar0 in prefetchable 64-bit mem space @ 0x00000000
    bar2 in prefetchable 64-bit mem space @ 0x20000000
    bar4 in 64-bit mem space @ 0xdff00000
    [32,0,1] type=MEM_CNTLR
    status=0x22b0 ( CAP 66MHZ FBTB DEVSEL=1 MSTR_ABORT_RCV )
    command=0x01d7 ( IO_ENABLE MEM_ENABLE MASTER_ENABLE WI_ENABLE
    PERR_ENABL
    E WC_ENABLE SERR_ENABLE )
    bar0 in prefetchable 64-bit mem space @ 0x01000000
    bar2 in prefetchable 64-bit mem space @ 0x01800000
    bar4 in prefetchable 64-bit mem space @ 0xdfec0000
    [32,0,2] type=MEM_CNTLR
    status=0x22b0 ( CAP 66MHZ FBTB DEVSEL=1 MSTR_ABORT_RCV )
    command=0x01d7 ( IO_ENABLE MEM_ENABLE MASTER_ENABLE WI_ENABLE
    PERR_ENABL
    E WC_ENABLE SERR_ENABLE )
    bar0 in 64-bit mem space @ 0x1c000000
    etc etc.


    Note the tuples there for [bus,device, function]. Use those on all
    your pci commands. You'll have to guess at the bus numbering in your
    system.

    Re the VME addressing, get used to using the sysVmeToLocalAdrs
    functions when you want VME addresses. VME address on your PPC are
    accessed throught the VME bridge, which makes them into PCI addresses,
    and then via a PCI bridge (DISCO, Raven, etc), which maps them to 60x
    (local addresses).

    When you think it works, try turing on some leds or something on the
    VME boards, and use the correct sys*Vme* function to translate the
    addresses.

    Re the vendor issue, if they are complaining about another vendors
    card, just put one of their cards in the offending slot, or call back
    and try to get another tech support person, or call the PMC vendors'
    support instead. One of those should work.

    Good luck,
    lc



    Jammer wrote:
    > Thanks very much for this info.
    > Since I posted this, I've talked to the Vendor and slightly more
    > understand what's going on. There's a "tempe" Tundra chip on the PCI #0
    > bus which is the VME bridge.
    > The DSS card is a PCI PMC card in one of the PMC slots on the SBC (it
    > has 2).
    >
    > Today and yesterday I did some testing, and it's sort of strange that I
    > can get the card mostly working in one of the PCI slots, but not in the
    > other one. I also have a ScramNet GT PMC card, and it also works
    > "funny" in this same slot, but works ok in the other slot.
    >
    > What I'm confused about it that I have 3 "Master" VME windows
    > configured within the PCI #0 space when I use sysMasterPortsShow(). I'm
    > not sure if that's correct or not. I can see how there needs to be
    > access to the tundra chip itself within this PCI #0 space, but I guess
    > I'm confused why the Master Windows need to be there, maybe becuase
    > they're really part of the Tundra chip? Shows you how little I know
    > about VME and the access to it.
    >
    > I am starting to make the distinction and understand PowerPC memory
    > access vs. the VME bus addresses. I'm only really concerned about the
    > addresses within my programming environment conflicting, meaning, what
    > the PowerPC sees (e.g., pointer values)
    >
    > The vendor is giving me a very hard time that we're trying to integrate
    > a "3rd party card" with thier board/BSP, where I say that any PCI card
    > plugged into the PMC slot should be "seen" by pciConfigTopoShow, which
    > is NOT the case for anything plugged into this specific PMC site.
    >
    > Thanks again.
    >
    > LarryC wrote:
    > > Hi:
    > >
    > > I think what might be confusing is that the VME is probably bridge via
    > > a Universe chip onto the PCI space. So, you should see VME addresses
    > > somewhere within your whole range of pci addresses. This is the most
    > > common way of doing things.
    > >
    > > Also, don't confuse VME addresses (ie those on the VME bus) with the
    > > CPUs address space, of which the PCI belongs. You have to do all the
    > > math to convert between them, or use the bsp functions
    > > sysBusToLocalAdrs (to trans vme to cpu), and sysLocalToBusAdrs to go
    > > the other way.
    > >
    > > I don't follow how your DSS card is installed. I'm guessing it is on a
    > > PMC site and not a VME card?
    > >
    > > The only problem I can recall causing your type of issues is that the
    > > TOTAL address range on the local bus allocated to PCI is exceeded due
    > > to expanding the VME address range such that all the PCI devices and
    > > vme address no longer fit. If that's really happening, then there
    > > should be some BSP #define that can increase the size of the total pci
    > > window that the can be used to accomodate all your pci devices, vme
    > > windows, etc.
    > >
    > > sysPciShow and sysVmeShow should help if they are in your BSP.
    > >
    > > Good luck,
    > > lc
    > > Jammer wrote:
    > > > Hello,
    > > >
    > > > I'm trying to migrate to a Synergy MANTA DX3 dual 7457 CPU SBC. I have
    > > > a DSS networks 5164 4-port GB Ethernet card I'm trying to get to work
    > > > on the board, under vxWorks 5.5.1, and it just won't work, can't get it
    > > > loaded into the MUX without locking the board up in "funny" ways. And
    > > > it depends on which port I try to load into the MUX as well, which
    > > > seems strange.
    > > >
    > > > In the past few days I had an unrelated VME Window problem (which I
    > > > think I now understand and have solved), and it just dawned on me, that
    > > > I don't think that you can have the board set up with PCI space, and
    > > > VME window(s) at the same memory addresses, right? This would mean the
    > > > VME windows and PCI would conflict I think, is that true?
    > > > I don't think I can even have the two PCI spaces overlapping, or that
    > > > would be extremely bad as well, right?
    > > >
    > > > For example, Synergy sets up 3 VME windows in the BSP by default, and I
    > > > remember at least 2 of them being at 0x90M and 0xa0M (don't have the
    > > > board in frot of me).
    > > > This board also has 2 PCI spaces, PCI0 and PCI1. PCI 0 was set up at
    > > > 0x80M with some size (either 256, 512, or 768 MB), and PCI1 was set up
    > > > at 0xb0M with 512MB size.
    > > > This appears that maybe the PCI spaces are even conflicting with
    > > > themselves. But on top of that, the VME window at 0xb0M would
    > > > definitely conflict with the DSS card in that specific slot. This makes
    > > > sense to me as why I can't get the card working, and it does "weird"
    > > > things.
    > > >
    > > > On top of all that, on the board pciConfigTopoShow won't show anything
    > > > on anothe bus but the PCI0 bus, which might be a sign of this
    > > > conflicting.
    > > >
    > > > Could anyone confirm that all these memory addresses MUST be different
    > > > and not overlap? If so that would give me some confidence in dealing
    > > > with Synergy.They aren't that helpful lately.



+ Reply to Thread