What still uses the block layer? - Kernel

This is a discussion on What still uses the block layer? - Kernel ; My impression from asking questions on the linux-scsi mailing list is that the scsi upper/middle/lower layers doesn't use the block layer described in Documentation/block/*. For example, the scsi guys say: http://marc.info/?l=linux-scsi&m=118633268527856&w=2 Instead of using the block layer, SCSI reinvents this ...

+ Reply to Thread
Page 1 of 5 1 2 3 ... LastLast
Results 1 to 20 of 91

Thread: What still uses the block layer?

  1. What still uses the block layer?

    My impression from asking questions on the linux-scsi mailing list is that the
    scsi upper/middle/lower layers doesn't use the block layer described in
    Documentation/block/*.

    For example, the scsi guys say:
    http://marc.info/?l=linux-scsi&m=118633268527856&w=2

    Instead of using the block layer, SCSI reinvents this particular wheel itself.
    There's a scsi "upper layer" that provides /dev nodes, scsi low-level
    drivers, and a gigantic glue layer in between call the "scsi midlayer" that's
    something like a networking stack, and is responsible for losing track of all
    your devices so that the one SATA disk hardwired into your laptop might be
    sda or sdc depending on whether or not you had a USB key plugged in when you
    booted up. Anyway, the block layer isn't between any of these three, that I
    can tell.

    Now that IDE disks have been rerouted through the scsi layer, SATA goes
    through the scsi layer, USB goes through the scsi layer, firewire goes
    through the scsi layer... What's left? It seems like everything but
    ramdisks have now been routed through the scsi layer. My laptop hasn't got a
    single SCSI device but it also hasn't got any block devices that don't show
    up as scsi.

    So what's still using the block layer? How do the scsi layers and the block
    layer relate? I'm confused! (This is normal for me, but still...)

    Rob
    --
    "One of my most productive days was throwing away 1000 lines of code."
    - Ken Thompson.
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  2. Re: What still uses the block layer?

    On Thu, Oct 11, 2007 at 08:11:21PM -0500, Rob Landley wrote:
    > My impression from asking questions on the linux-scsi mailing list is that the
    > scsi upper/middle/lower layers doesn't use the block layer described in
    > Documentation/block/*.


    Entirely incorrect.

    > Instead of using the block layer, SCSI reinvents this particular wheel itself.
    > There's a scsi "upper layer" that provides /dev nodes, scsi low-level
    > drivers, and a gigantic glue layer in between call the "scsi midlayer" that's
    > something like a networking stack, and is responsible for losing track of all
    > your devices so that the one SATA disk hardwired into your laptop might be
    > sda or sdc depending on whether or not you had a USB key plugged in when you
    > booted up. Anyway, the block layer isn't between any of these three, that I
    > can tell.


    You really need to get the **** over yourself.

    > Now that IDE disks have been rerouted through the scsi layer, SATA goes
    > through the scsi layer, USB goes through the scsi layer, firewire goes
    > through the scsi layer... What's left? It seems like everything but
    > ramdisks have now been routed through the scsi layer. My laptop hasn't got a
    > single SCSI device but it also hasn't got any block devices that don't show
    > up as scsi.


    That's nice. Why not take a look in drivers/block? Floppy, CCISS,
    CPQDA, UMEM, UBD, loop, NBD, SX8, UB, AoE, and many more.

    > So what's still using the block layer? How do the scsi layers and the block
    > layer relate? I'm confused! (This is normal for me, but still...)


    sd and sr are block drivers. In fact, the whole SCSI subsystem ...

    depends on BLOCK

    Just take a look at sd.c. The init code reads:

    for (i = 0; i < SD_MAJORS; i++)
    if (register_blkdev(sd_major(i), "sd") == 0)
    majors++;

    Then look at struct scsi_cmnd. It has a pointer to the block request
    that was passed down to it. struct scsi_device has a pointer to the
    block request_queue that's associated with the device. Block is what
    has elevators and io schedulers -- that work isn't duplicated by scsi.
    There's work to push more of scsi's infrastructure up into the block
    layer, so non-scsi block devices can take advantage of it.

    --
    Intel are signing my paycheques ... these opinions are still mine
    "Bill, look, we understand that you're interested in selling us this
    operating system, but compare it to ours. We can't possibly take such
    a retrograde step."
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  3. Re: What still uses the block layer?

    Matthew Wilcox wrote:
    > You really need to get the **** over yourself.


    That is so rude. You need to learn some manners.
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  4. Re: What still uses the block layer?

    David Newall wrote:
    > That is so rude.


    Such responses sometimes happen after provocative posts like the thread
    starter's. He could have asked straight away for help with fixing his
    boot environment instead of wrapping his question into a feigned design
    discussion. It appeared as if he is out for a fight rather than
    interested in help.
    --
    Stefan Richter
    -=====-=-=== =-=- -===-
    http://arcgraph.de/sr/
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  5. Re: What still uses the block layer?

    On Sat, 2007-10-13 at 16:05 -0600, Matthew Wilcox wrote:
    > On Thu, Oct 11, 2007 at 08:11:21PM -0500, Rob Landley wrote:
    > > My impression from asking questions on the linux-scsi mailing list is that the
    > > scsi upper/middle/lower layers doesn't use the block layer described in
    > > Documentation/block/*.

    >
    > Entirely incorrect.


    OK, right ... could we please get a sense of decorum back on this list.

    Rob, if you didn't ask your alleged questions in such a pejorative
    manner, we'd get a lot further; and Matthew, if you didn't rise to the
    bait so spectacularly it wouldn't prolong these threads.

    Really, both of you, I have better things to do with my time than
    mediate behaviours that should have been educated out of you in the
    kindergarten sand pit.

    James


    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  6. Re: What still uses the block layer?

    Am 14.10.2007 19:46 schrieb Stefan Richter:
    > David Newall wrote:
    >> That is so rude.

    >
    > Such responses sometimes happen after provocative posts like the thread
    > starter's.


    Provocation is often in the eye of the beholder, and basic manners
    should be observed nevertheless.

    > He could have asked straight away for help with fixing his
    > boot environment instead of wrapping his question into a feigned design
    > discussion.


    No, he couldn't have. He quite obviously didn't even know enough
    to understand his boot environment might be at fault, and hence
    was unable to conceive the question you're demanding from him.

    > It appeared as if he is out for a fight rather than
    > interested in help.


    It may have appeared like that from the highly antagonistic mindset
    that seems so prevalent in LKML. But if one just stepped back and
    took a breath before answering it should have been quite obvious
    that he wasn't. (out for a fight, that is)

    Granted, it can be difficult to comprehend the point of view of
    someone who does not know or understand something you yourself know
    or understand well. But you should at least be aware of that
    inability, and consequently refrain from accusing of provocation
    where there may be none. Hanlon's razor, cynical as it may sound at
    first, is an eminently humanistic principle.

    --
    Tilman Schmidt E-Mail: tilman@imap.cc
    Bonn, Germany
    Diese Nachricht besteht zu 100% aus wiederverwerteten Bits.
    Ungeöffnet mindestens haltbar bis: (siehe Rückseite)


    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.3rc1 (MingW32)
    Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

    iD8DBQFHEpnAMdB4Whm86/kRAg1wAJ9QKDnNPQjx1PlomYVsMbH6zSvr4gCffAVr
    /kqGZ0DLBKr0iUnyRcGW1s0=
    =AfvW
    -----END PGP SIGNATURE-----


  7. Re: What still uses the block layer?

    On Sunday 14 October 2007 12:46:12 pm Stefan Richter wrote:
    > David Newall wrote:
    > > That is so rude.


    When a reply contains as a reply to the first paragraph "you're wrong" with no
    elaboration, and as a reply to the second paragraph nothing but expletives
    and personal insults, I tend to stop reading. It really doesn't come across
    as a serious reply.

    I was at least attempting to ask a serious question.

    > Such responses sometimes happen after provocative posts like the thread
    > starter's. He could have asked straight away for help with fixing his
    > boot environment instead of wrapping his question into a feigned design
    > discussion. It appeared as if he is out for a fight rather than
    > interested in help.


    Actually, I was going through Documentation/block thinking about making a
    00-INDEX for it, but my earlier questions of the scsi guys left me with the
    impression that the block layer is _not_ used by the SCSI layer. And since
    every non-embedded modern storage device I'm aware of has been consumed by
    the SCSI layer (despite none of them actually having a discernably closer
    relationship to SCSI than ATA did), I didn't know whether or not it was more
    appropriate to index this directory or request its deletion. So I asked.

    Back when I asked the scsi guys about this, I got no direct answer. I
    asked "where does the block layer work into this" in the context of questiosn
    about the relationship between the scsi upper, middle, and lower layers, and
    I never got a reply, even though the question was quoted back at me here:
    http://www.mail-archive.com/linux-sc.../msg09086.html

    The closest I got to an answer was later in the thread:
    http://www.mail-archive.com/linux-sc.../msg09131.html

    Which said:
    > That approach makes the Linux block layer either a nuisance,
    > irrelevant or a complete anachronism (in the case of OSD).
    > IMO the linux block layer should be morphed into a library
    > of internal queue handling routines. Storage upper level
    > drivers such as sd can continue to present the "block"
    > view ** of storage devices such as disks.


    The gist of the thread (and the documentation I was referred to) is that the
    scsi "upper layer" presents /dev nodes and ioctls, the scsi mid-layer is a
    routing layer very roughly analogus to a TCP/IP stack, and the scsi low-layer
    drivers interface with specific pieces of hardware. Apparently, the block
    layer is not between any of these, they talk directly to each other. This
    would seem to indicate that I/O requests made to scsi devices are never
    routed through a common block I/O request handling layer shared with non-SCSI
    block devices. I was not, however, certain of this, hence my attempt to
    bring the topic back up.

    Oh, and sending a patch correcting Jens Axboe's address in this old
    documentation. He's apparently at Oracle now...

    Rob
    --
    "One of my most productive days was throwing away 1000 lines of code."
    - Ken Thompson.
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  8. Re: What still uses the block layer?

    On Sunday 14 October 2007 5:24:32 pm James Bottomley wrote:
    > On Sat, 2007-10-13 at 16:05 -0600, Matthew Wilcox wrote:
    > > On Thu, Oct 11, 2007 at 08:11:21PM -0500, Rob Landley wrote:
    > > > My impression from asking questions on the linux-scsi mailing list is
    > > > that the scsi upper/middle/lower layers doesn't use the block layer
    > > > described in Documentation/block/*.

    > >
    > > Entirely incorrect.

    >
    > OK, right ... could we please get a sense of decorum back on this list.


    Did I reply to the insult?

    > Rob, if you didn't ask your alleged questions in such a pejorative
    > manner, we'd get a lot further


    I'm not attempting to be pejorative.

    I admit a certain amount of personal annoyance that once the SCSI layer
    consumes a category of device (USB, SATA, PATA), they can often _only_ be
    used by going through the SCSI midlayer. (This strikes me as analogous to
    TCP/IP claiming ethernet and PPP devices so thoroughly that you can no longer
    address them as eth1 or /dev/ttyS0.)

    This has the annoying effect of bundling together different types of devices
    and making device enumeration unnecessarily difficult: my laptop only has one
    SATA hard drive and can't gain another without a soldering iron, but that
    drive could move from /dev/sda to /dev/sdb if I reboot the system with a USB
    key plugged in. This seems like a regrettable loss of orthogonality to me.
    I remember back when /dev/usb0 and /dev/hda were separate devices that showed
    up in /dev, but these days "it's SCSI" seems to trump "it's USB", "it's ATA",
    or "it's SATA". (Even though none of those are actually SCSI hardware, they
    just send a similar packet protocol across the wire.)

    The fact that udev can theoretically unwind this hairball is not an excuse for
    conflating different categories of devices in the first place. Avoiding an
    unnecessary problem seems superior to trying to get udev to solve it. Note
    that Ubuntu 7.04 solves it by sticking a UUID on every _partition_, and then
    spinning up my external USB hard drive trying to find the root partition on a
    reboot. Tell me how this can be considered progress:

    > # /etc/fstab: static file system information.
    > #
    > #
    > proc /proc proc defaults 0 0
    > # /dev/sda1
    > UUID=04d1b984-bd65-46f1-9a77-c158cf4bed1b / ext3

    defaults,errors=remount-ro,noatime 0 1
    > # /dev/sda5
    > UUID=cdf0936d-9f19-42c6-b131-9fefcf1321ef none swap sw

    0 0
    > /dev/scd0 /media/cdrom0 udf,iso9660 user,noauto 0 0
    > UUID=86bbb512-ab7e-4a12-8618-1190f032c082 /boot ext3 defaults 0 0


    Conflating categories of hardware that cannot easily be enumerated (USB) with
    categories that can (the SATA hard drive in my laptop, of which there can be
    only one) strikes me as a bad thing. Putting them in a common "scsi device
    pool" within which they do not enumerate consistently is not something I
    enjoy dealing with.

    However, the response to my attempts to express this dissatisfaction on the
    SCSI list a few months ago came too close to a flamewar for me to consider
    continuing it productive. I'd still love to update the "2.4 scsi howto" and
    corresponding sg howto, but lack the expertise. The SCSI layer really isn't
    my area, and I was much happier back when I could avoid using it at all.

    The question I was trying to ask _here_ was about the block layer. I seem not
    to have asked it very well. Sorry 'bout that.

    Rob
    --
    "One of my most productive days was throwing away 1000 lines of code."
    - Ken Thompson.
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  9. Re: What still uses the block layer?

    --- James Bottomley wrote:
    > On Sat, 2007-10-13 at 16:05 -0600, Matthew Wilcox wrote:
    > > On Thu, Oct 11, 2007 at 08:11:21PM -0500, Rob Landley wrote:
    > > > My impression from asking questions on the linux-scsi mailing list is that the
    > > > scsi upper/middle/lower layers doesn't use the block layer described in
    > > > Documentation/block/*.

    > >
    > > Entirely incorrect.

    >
    > OK, right ... could we please get a sense of decorum back on this list.
    >
    > Rob, if you didn't ask your alleged questions in such a pejorative
    > manner, we'd get a lot further; and Matthew, if you didn't rise to the
    > bait so spectacularly it wouldn't prolong these threads.
    >
    > Really, both of you, I have better things to do with my time than
    > mediate behaviours that should have been educated out of you in the
    > kindergarten sand pit.


    I really didn't find Rob's email "pejorative" at all. It seems to me
    he was just asking for clarification, information and trying to
    understand how it all works and ties together. His email seemed
    genuine enough of a person just asking to understand how it all works.

    Matthew's expletive and extremely rude response really shows
    the general attitude of the linux-scsi people.

    Heck, I got a similar response just a week ago here on the
    list, trying to convince Garzik and his band, that storage nodes
    SHOULD NOT be SAS WWN generators. Should I have even tried? That's
    the question.

    Good luck everyone,
    Luben

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  10. Re: What still uses the block layer?

    On Sunday October 14, rob@landley.net wrote:
    > On Sunday 14 October 2007 12:46:12 pm Stefan Richter wrote:
    > > David Newall wrote:
    > > > That is so rude.

    >
    > When a reply contains as a reply to the first paragraph "you're wrong" with no
    > elaboration, and as a reply to the second paragraph nothing but expletives
    > and personal insults, I tend to stop reading. It really doesn't come across
    > as a serious reply.
    >
    > I was at least attempting to ask a serious question.


    Indeed you were, and let me try to answer it as best I can.

    I like to think of the "block layer" as two main parts.

    Firstly there is the "interface" which it defines, embodied primarily
    in generic_make_request() and 'struct bio'. There are various other
    small routines in ll_rw_blk.c, and there is 'struct request_queue'
    which is also involved in the other half of the block layer.

    This interface defines how requests are passed down, how their
    completion is acknowledged, and various other little details

    Any block device can register a make_request_fn function and get the
    requests (struct bio) almost exactly as the client (filesystem or
    whatever) sent them down - just with a few sanity checks and some
    translation (for partitions) applied.

    The other half of the "block layer" is the io scheduler code.
    This involves the 'struct request' and __make_request() and the various
    routines it calls.
    This collects bios (passed down from clients) and produces 'requests'
    which devices can handle. One of the important differences between
    bios and requests is the amount of parallelism.
    A filesystem can send down as may concurrent bios as it likes (or as
    it can allocate memory for).
    A device can only handle a limited number of requests at a time,
    depending on the limit of the 'tags command queueing' mechanism
    particular to that device.
    The scheduler bridges this parallelism gap by .... scheduling.

    So the "block layer" consists of "block interface" and "io scheduler"

    All block devices use the "block interface" - they have no choice.
    Many block devices use the "io scheduler", but many don't.
    md and dm, loop, umem, and others do their own scheduling as they have
    needs that are specific to the devices, or that otherwise don't
    benefit from the io scheduler (which is really designed for
    rotating-media style devices).

    SCSI devices can be both block device and non-block devices
    (traditionally 'char devices').

    The 'scsi generic' or 'sg' interface to SCSI devices allows arbitrary
    SCSI commands to be sent to a SCSI device. There are many SCSI
    devices that are not block devices as all (media robots, etc).

    When a SCSI device is being used as a block device, the block
    interface is used. When it is being used as a 'generic device', the
    block interface is not used.

    Now we get to the heart of the matter, and to where my knowledge
    becomes a little less detailed - so please forgive if I say something
    silly.

    I believe that the SCSI-generic handling still uses the IO scheduler,
    even though it doesn't use the block interface.
    It is probable that the IO scheduler is not a perfect match for the
    needs of SCSI-generic handling. Given it's origin, that should not be
    surprising.

    I believe the linux-scsi email that you referred was addressing this
    issue. When the author says:

    That approach makes the Linux block layer either a nuisance,
    irrelevant or a complete anachronism

    I believe he is referring to what I would call the IO scheduler, and is
    observing that it is not a perfect fit. He is probably right.

    So to answer your question:

    SCSI block devices use both the "block interface" and the "io
    scheduler" and I believe that when people talk about "the block layer"
    they refer to these two things.
    i.e. the SCSI layer provides "scsi_request_fn". The block interface
    calls __make_request which performs IO scheduling and calls
    scsi_request_fn for each request.

    Hope that helps.

    NeilBrown
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  11. Re: What still uses the block layer?

    On Sun, Oct 14, 2007 at 06:45:44PM -0500, Rob Landley wrote:
    > I admit a certain amount of personal annoyance that once the SCSI
    > layer consumes a category of device (USB, SATA, PATA), they can
    > often _only_ be used by going through the SCSI midlayer. (This
    > strikes me as analogous to TCP/IP claiming ethernet and PPP devices
    > so thoroughly that you can no longer address them as eth1 or
    > /dev/ttyS0.)


    That's because modern USB, ATAPI (what was once known as IDE), SATA
    really *all* using the SCSI command protocols at the low level, just
    as Ethernet and PPP interfaces really are fundamentally the same
    thing. You can rail against it, but that's the mark of someone who
    refuses to accept reality.

    > This has the annoying effect of bundling together different types of
    > devices and making device enumeration unnecessarily difficult: my
    > laptop only has one SATA hard drive and can't gain another without a
    > soldering iron, but that drive could move from /dev/sda to /dev/sdb
    > if I reboot the system with a USB key plugged in. This seems like a
    > regrettable loss of orthogonality to me. I remember back when
    > /dev/usb0 and /dev/hda were separate devices that showed up in /dev,
    > but these days "it's SCSI" seems to trump "it's USB", "it's ATA", or
    > "it's SATA". (Even though none of those are actually SCSI hardware,
    > they just send a similar packet protocol across the wire.)


    You're showing your ignorance here. In fact in the past few years,
    ATA and SCSI has been converging significantly, with the ATAPI
    specification has essentially incorporating the SCSI protocol by
    reference and by value --- with the point that SAS was developed by
    the SCSI Trade Association, and SAS is effectively a superset of SATA,
    to the point where with care, you can actually mix SAS and SATA drives
    on the same in enclosure (SAS and SATA are physically compatible on
    the connector level).

    More to the point, with SATA, hot plugging has been designed in, so
    probing order is not going to be well defined, just as with USB
    devices. And there are already relatively common situations where the
    same disk can show up via multiple different interfaces.

    For example, if you have a modern Thinkpad with an secondary SATA hard
    drive in an Ultrabay, and you plug it into the Ultrabay in your T60,
    it will show up as a SATA drive. However, if you plug it into the
    Advanced dock, it shows up as a USB device. And with iSCSI not only
    can you encapsulate a SCSI command stream over USB, you can do so over
    IP as well. In any case, regardless of how the physical SATA drive is
    attached to the system, you want it to show up as the same device and
    be mounted in the same location.

    That's why identifying filesystem by UUID's or Labels is so critical.
    This is not a new concept; we've had the capability to do this for
    over a decade, and I always knew it would be necessary for us to do
    this sooner or later --- which is why I added the UUID support to ext2
    back in 1996.

    > The fact that udev can theoretically unwind this hairball is not an
    > excuse for conflating different categories of devices in the first
    > place.


    See the thinkpad Ultrabay drive example above. You address hosts by
    IP address; it doesn't matter whether you access them via a PPP
    interface, or a wireless interface, or a ethernet interface.
    Similarly, a disk could in theory be accessible over USB, SATA, or
    iSCSI, and the Thinkpad example is only one such where the same
    filesystem might be accessible over multiple interfaces. And with
    multipath fiber channel SAN's (and I hate to break it to you, but FC
    also uses SCSI protocols) storage is very much looking more and more
    like networking.

    - Ted
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  12. Re: What still uses the block layer?

    Rob Landley wrote:
    > I was at least attempting to ask a serious question.

    ....
    > Actually, I was going through Documentation/block thinking about making a
    > 00-INDEX for it, but my earlier questions of the scsi guys left me with the
    > impression that the block layer is _not_ used by the SCSI layer.


    Ah, so it was about your documentation work. I already forgot the
    context of your previous inquiries. Alas the tone of them already did
    some damage, leading to responses like these.

    ....
    > since
    > every non-embedded modern storage device I'm aware of has been consumed by
    > the SCSI layer (despite none of them actually having a discernably closer
    > relationship to SCSI than ATA did)

    ....

    The Linux SCSI subsystems don't consume, they provide services; nowadays
    not only for SCSI hardware and SCSI protocols but also for a number of
    subsystems whose tasks are similar enough to SCSI subsystems to make the
    SCSI core and upper SCSI layer useful to them too.

    BTW:
    | Now that IDE disks have been rerouted through the scsi layer, SATA goes
    | through the scsi layer, USB goes through the scsi layer, firewire goes
    | through the scsi layer...

    As a side note, SBP-2 is a SCSI transport protocol, hence ieee1394/sbp2
    and firewire/fw-sbp2 are Linux SCSI low-level drivers. Anything else
    would be just wrong and infeasible in this particular case.
    --
    Stefan Richter
    -=====-=-=== =-=- -====
    http://arcgraph.de/sr/
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  13. Re: What still uses the block layer?

    On Sun, Oct 14, 2007 at 06:45:44PM -0500, Rob Landley wrote:
    > On Sunday 14 October 2007 5:24:32 pm James Bottomley wrote:
    > > On Sat, 2007-10-13 at 16:05 -0600, Matthew Wilcox wrote:
    > > > On Thu, Oct 11, 2007 at 08:11:21PM -0500, Rob Landley wrote:
    > > > > My impression from asking questions on the linux-scsi mailing list is
    > > > > that the scsi upper/middle/lower layers doesn't use the block layer
    > > > > described in Documentation/block/*.
    > > >
    > > > Entirely incorrect.

    > >
    > > OK, right ... could we please get a sense of decorum back on this list.

    >
    > Did I reply to the insult?
    >
    > > Rob, if you didn't ask your alleged questions in such a pejorative
    > > manner, we'd get a lot further

    >
    > I'm not attempting to be pejorative.
    >
    > I admit a certain amount of personal annoyance that once the SCSI layer
    > consumes a category of device (USB, SATA, PATA), they can often _only_ be
    > used by going through the SCSI midlayer. (This strikes me as analogous to
    > TCP/IP claiming ethernet and PPP devices so thoroughly that you can no longer
    > address them as eth1 or /dev/ttyS0.)


    If you hate USB storage devices using scsi, please use the ub driver,
    that is what it was written for.

    > This has the annoying effect of bundling together different types of devices
    > and making device enumeration unnecessarily difficult: my laptop only has one
    > SATA hard drive and can't gain another without a soldering iron, but that
    > drive could move from /dev/sda to /dev/sdb if I reboot the system with a USB
    > key plugged in. This seems like a regrettable loss of orthogonality to me.
    > I remember back when /dev/usb0 and /dev/hda were separate devices that showed
    > up in /dev, but these days "it's SCSI" seems to trump "it's USB", "it's ATA",
    > or "it's SATA". (Even though none of those are actually SCSI hardware, they
    > just send a similar packet protocol across the wire.)


    When did usb-storage devices ever show up as /dev/usb0? USB flash disks
    are really SCSI devices, look at the USB storage spec for proof of that.

    thanks,

    greg k-h
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  14. Re: What still uses the block layer?

    On Sunday 14 October 2007 7:45:46 pm Luben Tuikov wrote:
    > Matthew's expletive and extremely rude response really shows
    > the general attitude of the linux-scsi people.


    No, it doesn't. James Bottomley has been exceedingly polite and helpful, as
    were several other people on the linux-scsi list when I asked them about this
    stuff back in August.

    Religion, politics, and anything remotely related to hotplug appear to be
    topics to avoid in polite company if you want it to remain polite. (My
    gripes with scsi mostly have to do with device enumeration. My attempts to
    use sysfs also have to do with device enumeration. I've spotted a trend
    here.)

    Rob
    --
    "One of my most productive days was throwing away 1000 lines of code."
    - Ken Thompson.
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  15. Re: What still uses the block layer?

    On Sunday 14 October 2007 8:45:03 pm Theodore Tso wrote:
    > On Sun, Oct 14, 2007 at 06:45:44PM -0500, Rob Landley wrote:
    > > I admit a certain amount of personal annoyance that once the SCSI
    > > layer consumes a category of device (USB, SATA, PATA), they can
    > > often _only_ be used by going through the SCSI midlayer. (This
    > > strikes me as analogous to TCP/IP claiming ethernet and PPP devices
    > > so thoroughly that you can no longer address them as eth1 or
    > > /dev/ttyS0.)

    >
    > That's because modern USB, ATAPI (what was once known as IDE), SATA
    > really *all* using the SCSI command protocols at the low level,


    Ok, I'll bite. If it's all "real" scsi, why does ioctl(SG_EMULATED_HOST)
    exist? exist if it's all "real" scsi?

    > just
    > as Ethernet and PPP interfaces really are fundamentally the same
    > thing.


    They're the same thing?

    Do you mean that on a system with both, going:
    ifconfig eth1 66.92.53.140
    ifconfig ppp 192.168.0.42

    Would be functionally equivalent to:
    ifconfig eth1 192.168.0.42
    ifconfig ppp 66.92.53.140

    So if on one boot the addresses are assigned the first way, and upon reboot
    they're assigned in the second way by exact the same set of commands... well
    that's not IMPORTANT, is it? (Or is it that everyone everywhere should use
    dhcp for everything, and static addressing is obsolete and no longer
    supported? Apparently dhcp addresses should be delivered by machines with
    only one network interface of any type...)

    This is my objection. Even when enumerating multiple devices of the same type
    is tricky, enumerating multiple devices of _different_ types should not be.
    There's a great big type indicator that is being _deliberately_ ignored, and
    large classes of devices (millions of laptops) where you know there's only
    going to be _one_ instance of a given type.

    By the way, ethernet cards contain a unique MAC address. Hard drives do not
    seem to, or if they do it's not being consistently exposed in a way I can
    find. This is sad. (No, reading data from the device to determine this gets
    us back to the "spinning up the external USB drive to find my root partition"
    gripe mentioned earlier.)

    > You can rail against it, but that's the mark of someone who
    > refuses to accept reality.


    Let me clarify: I'm talking about device enumeration.

    I've never had trouble enumerating a device that was _not_ routed through the
    scsi layer, largely because the systems I work with don't usually have more
    than one device of the same type. (There are millions of laptop and desktop
    devices out there where this is the common case. As I said, I may have four
    USB ports and the ability to plug hubs into them, but you can't add another
    SATA hard drive to my laptop without a soldering iron.)

    However, as soon as a device _is_ routed through the scsi layer (as PATA was a
    few versions back), it gets conflated with numerous other devices. This
    creates problems. SATA isn't hard to enmerate in my laptop, USB potentially
    is. Dumpting all the SATA devices into the same bucket with the USB devices
    makes both harder to enumerate.

    > > This has the annoying effect of bundling together different types of
    > > devices and making device enumeration unnecessarily difficult: my
    > > laptop only has one SATA hard drive and can't gain another without a
    > > soldering iron, but that drive could move from /dev/sda to /dev/sdb
    > > if I reboot the system with a USB key plugged in. This seems like a
    > > regrettable loss of orthogonality to me. I remember back when
    > > /dev/usb0 and /dev/hda were separate devices that showed up in /dev,
    > > but these days "it's SCSI" seems to trump "it's USB", "it's ATA", or
    > > "it's SATA". (Even though none of those are actually SCSI hardware,
    > > they just send a similar packet protocol across the wire.)

    >
    > You're showing your ignorance here.


    I have buckets of ignorance. It's why I ask questions.

    > In fact in the past few years,
    > ATA and SCSI has been converging significantly,


    And down far enough all these devices are powered by electricity. Are we
    going to wind up with /dev/electric[1-999]?

    SATA != PATA != USB. But /dev/sda can be PATA, /dev/sdb SATA, and /dev/sdc
    USB. And they can move relative to each other. This didn't used to be the
    case. Why is it considered an improvement?

    > with the ATAPI
    > specification has essentially incorporating the SCSI protocol by
    > reference and by value --- with the point that SAS was developed by
    > the SCSI Trade Association, and SAS is effectively a superset of SATA,
    > to the point where with care, you can actually mix SAS and SATA drives
    > on the same in enclosure (SAS and SATA are physically compatible on
    > the connector level).


    I'm aware of this, and under the impression they're both modified gigabit
    ethernet at the PHY level. Should the hard drive become eth2?

    > More to the point, with SATA, hot plugging has been designed in, so
    > probing order is not going to be well defined,


    The spec may define the capability to hotplug, but your average laptop doesn't
    not offer the capability to hotplug anything into its SATA controllers. The
    hard drive is screwed in (due to the portability part of laptopness), all the
    controllers wired onto the motherboard are accounted for, none are exposed
    externally. What _is_ exposed externally is USB, and if you want to add an
    extra hard drive you can buy a cheap USB one at Fry's.

    In such a case, which is common, the first SATA hard drive is reliably the
    disk containing the root partition, and there's no need to stick a UUID
    in /etc/fstab.

    The problem is, "the first SATA hard drive" is not a stable identifier in a
    system where SATA and USB devices are dumped in the same bucket and given a
    big stir. Dumping SATA and USB devices into the same bucket (because they
    smell a bit like SCSI) is what I am objecting to.

    > just as with USB
    > devices. And there are already relatively common situations where the
    > same disk can show up via multiple different interfaces.


    It was also possible to buy a hotplug PATA ide enclosure. So what? The vast
    majority of traditional IDE users happily ignored this, and went on with
    their lives.

    > For example, if you have a modern Thinkpad with an secondary SATA hard
    > drive in an Ultrabay, and you plug it into the Ultrabay in your T60,
    > it will show up as a SATA drive.


    I remember the config option about enumerating onboard IDE controllers first.
    It didn't really matter what order they were enumerated in as long as it was
    controllable.

    Presumably if the primary SATA hard drive was /dev/sata and the slot
    with "secondary" in its name got /dev/satb, life would be good. And the
    presence or absence of /dev/satb wouldn't affect USB devices and such if they
    weren't in the same namespace.

    > However, if you plug it into the
    > Advanced dock, it shows up as a USB device.


    You plug it in somewhere else, it shows up somewhere else. This sounds
    familiar to old IDE users.

    How is it harder for udev to make a stable symlink for this drive that
    sometimes points to /dev/satb and sometimes to /dev/usb1? (Harder than a
    symlink that sometimes points to /dev/sdb and sometimes to /dev/sdd? You
    don't have persistent naming _now_, so the objection seems to be that
    maintaining the distinction between device types would not be a perfect
    solution in all cases. I agree. So?)

    > And with iSCSI not only
    > can you encapsulate a SCSI command stream over USB, you can do so over
    > IP as well.


    Yup. And you've been able to make a network block device for years. They
    showed up as /dev/nd0, a distinct type of block device which you (and your
    scripts) could find. Now yet another way of doing the same thing is mixed
    into the same scsi bucket and given a stir...

    > In any case, regardless of how the physical SATA drive is
    > attached to the system, you want it to show up as the same device and
    > be mounted in the same location.


    If my laptop's hard drive reliably showed up as /dev/sda every time, and I
    could count on that, I wouldn't be complaining about it. The entire problem
    is that it isn't guaranteed to do that, and thus /etc/fstab is a nightmare I
    can't edit.

    You could meet this definition of "the same" by having every block device in
    the system show up as /dev/block[a-z] no matter what type it was, and all the
    char devices show up as /dev/char[aa-zz], shuffle them all each reboot, and
    then have all the programs iterate through all of them any time they wanted
    something specific.

    I'm rather glad that /dev/ttyS0 and /dev/zero aren't easy to mix up.

    > That's why identifying filesystem by UUID's or Labels is so critical.
    > This is not a new concept; we've had the capability to do this for
    > over a decade, and I always knew it would be necessary for us to do
    > this sooner or later --- which is why I added the UUID support to ext2
    > back in 1996.


    It's necessary for IBM big iron to do this. It's generally not necessary for
    laptops or embedded systems to do this if they distinguish between _types_ of
    devices, which is something they until recently did for the types of devices
    I was interested in, and something they _stopped_ doing when everything got
    merged into the scsi layer, and I consider this a regression.

    No, distinguishing between types of devices is not a perfect solution to
    device enumeration, but it was sufficient for all my use cases for many
    years, and would still be if the kernel still did it, and I'm not alone here.

    > > The fact that udev can theoretically unwind this hairball is not an
    > > excuse for conflating different categories of devices in the first
    > > place.

    >
    > See the thinkpad Ultrabay drive example above.


    Last week I drove my laptop so deep into swap (with a "make -j" on qemu) that
    after half an hour trying to repaint my kmail window, it locked solid.
    Again. You'd think the oom killer would come to the rescue, but it didn't.
    Maybe Ubuntu disabled it. I have _2_gigs_ of ram in this sucker, on a stock
    Ubuntu 7.04 install (with the "upgrade all" tab pressed a few times), and yet
    I managed to make it swap itself to death one more time.

    Virtual memory isn't perfect. I've _always_ been able to come up with
    examples where it just doesn't work for me. This doesn't mean VM overcommit
    should be abolished, because it's useful more often than not.

    So you have a counterexample. Ok. I can't actually see how your
    counterexample would be worse off than it is now; just differently worse off.

    > You address hosts by
    > IP address; it doesn't matter whether you access them via a PPP
    > interface, or a wireless interface, or a ethernet interface.


    It does when I'm configuring the interfaces.

    > Similarly, a disk could in theory be accessible over USB, SATA, or
    > iSCSI, and the Thinkpad example is only one such where the same
    > filesystem might be accessible over multiple interfaces. And with
    > multipath fiber channel SAN's (and I hate to break it to you, but FC
    > also uses SCSI protocols) storage is very much looking more and more
    > like networking.


    And in the networking world I'm able to say that this local machine has a
    static IP that is not world-routable. It is separate, manually configured, I
    put it _right_here_, and I personally know that it's not going to move
    because I'm the one who put it there and I'm the only one who would move it.

    Over on the networking side of things I can "ifconfig lo 127.0.0.1" without
    first probing all the interfaces to figure out which one's loopback and which
    one's the wireless card.

    I note that the eth0 and eth1 names are dynamically assigned on a first come
    first serve basis (like scsi). This never causes me a problem because the
    driver loading order is constant, and once you figure out that eth0 is
    gigabit and eth1 is the 80211g it _stays_ that way across reboots, reliably.
    Yeah, it's a heuristic. Hands up everybody relying on such a heuristic in
    the real world.

    Possibly one solution here is to document that the SATA drivers load before
    any other scsi device, and the driver subsystem _waits_ for that to finish
    enumerating before trying any other kind of scsi device, with a barrier of
    some kind), and then any SATA devices present at boot time will reliably get
    those names in that order (no races, no variation) and anything after that is
    a separate problem. (Of course this would involve making it true if it
    currently isn't. It's still a mess to dump all sorts of different devices in
    the same namespace, but at least for the common case of a laptop with a SATA
    root partition this would let us get the UUID out of /etc/fstab).

    > - Ted


    Rob
    --
    "One of my most productive days was throwing away 1000 lines of code."
    - Ken Thompson.
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  16. OOM killer gripe (was Re: What still uses the block layer?)

    On Monday 15 October 2007 18:04, Rob Landley wrote:
    > On Sunday 14 October 2007 8:45:03 pm Theodore Tso wrote:


    > > > excuse for conflating different categories of devices in the first
    > > > place.

    > >
    > > See the thinkpad Ultrabay drive example above.

    >
    > Last week I drove my laptop so deep into swap (with a "make -j" on qemu)
    > that after half an hour trying to repaint my kmail window, it locked solid.
    > Again. You'd think the oom killer would come to the rescue, but it didn't.
    > Maybe Ubuntu disabled it. I have _2_gigs_ of ram in this sucker, on a
    > stock Ubuntu 7.04 install (with the "upgrade all" tab pressed a few times),
    > and yet I managed to make it swap itself to death one more time.
    >
    > Virtual memory isn't perfect. I've _always_ been able to come up with
    > examples where it just doesn't work for me. This doesn't mean VM
    > overcommit should be abolished, because it's useful more often than not.


    I hate to go completely offtopic here, but disks are so incredibly
    slow when compared to RAM that there is really nothing the kernel
    can do about this. Presumably the job will finish, given infinite
    time.

    How much swap do you have configured? You really shouldn't configure
    so much unless you do want the kernel to actually use it all, right?
    Because if we're not really conservative about OOM killing, then the
    user who actually really did want to use all the swap they configured
    gets angry when we kill their jobs without using it all.

    Would an oom-kill-someone-now sysrq be of help, I wonder?
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  17. Re: What still uses the block layer?

    --- Rob Landley wrote:
    > On Sunday 14 October 2007 7:45:46 pm Luben Tuikov wrote:
    > > Matthew's expletive and extremely rude response really shows
    > > the general attitude of the linux-scsi people.

    >
    > No, it doesn't. James Bottomley has been exceedingly polite and helpful, as
    > were several other people on the linux-scsi list when I asked them about this
    > stuff back in August.


    I wasn't referring to him specifically. He also stepped into the WWN
    thread in the same manner as he did in your thread.

    > Religion, politics, and anything remotely related to hotplug appear to be
    > topics to avoid in polite company if you want it to remain polite. (My
    > gripes with scsi mostly have to do with device enumeration. My attempts to
    > use sysfs also have to do with device enumeration. I've spotted a trend
    > here.)
    >
    > Rob
    > --
    > "One of my most productive days was throwing away 1000 lines of code."
    > - Ken Thompson.
    >


    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  18. Re: What still uses the block layer?

    On Sun, Oct 14, 2007 at 11:00:15PM -0700, Greg KH wrote:
    > If you hate USB storage devices using scsi, please use the ub driver,
    > that is what it was written for.


    The ub driver is a really dumb piece of ****. It only drivers usb storage
    devices using a scsi protocol set, and duplicates the scsi stack in a very
    suboptimal way.

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  19. Re: What still uses the block layer?

    On 10/15/07, Rob Landley wrote:
    > I note that the eth0 and eth1 names are dynamically assigned on a first come
    > first serve basis (like scsi). This never causes me a problem because the
    > driver loading order is constant, and once you figure out that eth0 is
    > gigabit and eth1 is the 80211g it _stays_ that way across reboots, reliably.
    > Yeah, it's a heuristic. Hands up everybody relying on such a heuristic in
    > the real world.


    Umm, not quite, from my experiences with pre-production wireless
    drivers, (another story, another time) fancy stuff is being done in
    udev to make sure that your gigabit card is always assigned to eth0.

    --

    Julian Calaby

    Email: julian.calaby@gmail.com
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  20. Re: What still uses the block layer?

    On Monday 15 October 2007 12:44:19 am Stefan Richter wrote:
    > Rob Landley wrote:
    > > I was at least attempting to ask a serious question.

    >
    > ...
    >
    > > Actually, I was going through Documentation/block thinking about making a
    > > 00-INDEX for it, but my earlier questions of the scsi guys left me with
    > > the impression that the block layer is _not_ used by the SCSI layer.

    >
    > Ah, so it was about your documentation work.


    Well, triggered by. (This documentation stuff makes me poke into corners of
    the kernel I ordinarily otherwise avoid, for various reasons. I don't
    currently have the luxury of saying "beats me how this bit works, not my
    area".)

    > I already forgot the
    > context of your previous inquiries. Alas the tone of them already did
    > some damage, leading to responses like these.


    Sorry about that. My social skills are finite, I tend to exhaust them when I
    do too much at once.

    The resulting documentation should be very polite and apolitical.

    > > since
    > > every non-embedded modern storage device I'm aware of has been consumed
    > > by the SCSI layer (despite none of them actually having a discernably
    > > closer relationship to SCSI than ATA did)

    >
    > The Linux SCSI subsystems don't consume, they provide services; nowadays
    > not only for SCSI hardware and SCSI protocols but also for a number of
    > subsystems whose tasks are similar enough to SCSI subsystems to make the
    > SCSI core and upper SCSI layer useful to them too.


    This discussion has clarified for me that my objection isn't the scsi layer
    itself, it's the /dev/sd? namespace combining devices that would otherwise
    be /dev/hda, /dev/nd0, /dev/ub0 (or usb0 or some such), and /dev/sata into a
    single linear namespace that's unreliably ordered.

    > BTW:
    > | Now that IDE disks have been rerouted through the scsi layer, SATA goes
    > | through the scsi layer, USB goes through the scsi layer, firewire goes
    > | through the scsi layer...
    >
    > As a side note, SBP-2 is a SCSI transport protocol, hence ieee1394/sbp2
    > and firewire/fw-sbp2 are Linux SCSI low-level drivers. Anything else
    > would be just wrong and infeasible in this particular case.


    My "scsi mid layer" vs "block layer" question was about whether I should read
    up on the block layer if the scsi mid layer didn't use it. Neil Brown just
    sent me a nice email (which I'll have to reread in the morning when I'm more
    awake) that helps there.

    The "ide/sata/usb/firewire->scsi" complaint didn't belong in the same email as
    the original question, it's a line of questioning I put on hold on linux-scsi
    back in August when the thread started getting a bit heated for my tastes.

    To clarify, I think that merging ide, sata, usb, firewire, and others into a
    single device namespace causes each type of device to inherit that
    namespace's cumulative ordering issues, which is a bad thing. I have no real
    attachment to the underlying scsi or block layers. I've never seriously
    worked on either (although I'm trying to understand both).

    For example, usb devices are never easy to order. IDE devices (back when they
    had their own namespace) were trivial to order back when /dev/hda couldn't
    move without use of a screwdriver. USB and IDE devices are very different in
    that it's not possible to plug a USB device into an IDE controller (not
    without one _heck_ of an adapter) and vice versa. USB devices usually live
    outside the computer's case, and IDE devices inside the case. They're not
    the same thing.

    Combining USB and IDE into the same /dev/sd? namespace makes enumerating the
    IDE devices much harder than in the traditional "/dev/hdb doesn't move
    without a screwdriver" model. The merger creates a new problem for IDE, one
    which didn't exist before: the addition or removal of other unrelated types
    of devices may change this device's location next boot. It may be possible
    to add additional complication to the system to compensate, but what was the
    advantage of merging the namespaces in the first place?

    Rob
    --
    "One of my most productive days was throwing away 1000 lines of code."
    - Ken Thompson.
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

+ Reply to Thread
Page 1 of 5 1 2 3 ... LastLast