File System, Reincarnation Server and Process Manager - Minix

This is a discussion on File System, Reincarnation Server and Process Manager - Minix ; Hello, Reading through the Minix book, I learned that ever process besides the Process Manager (PM) is is a child (or descendant) of the Reincanration Server (RS). My question: Is this true for the File Server (FS) too? Reading through ...

+ Reply to Thread
Results 1 to 11 of 11

Thread: File System, Reincarnation Server and Process Manager

  1. File System, Reincarnation Server and Process Manager

    Hello,

    Reading through the Minix book, I learned that ever process besides
    the Process Manager (PM) is is a child (or descendant) of the
    Reincanration Server (RS). My question: Is this true for the File
    Server (FS) too?

    Reading through the book, some times it seems like the PM and the FS
    are the most fundamental proceses in Minix 3 (after the kernel).
    Scanning the source code, they seem to have a bit of a special role,
    and I notice in Figure 2-31 that the FS appears to be loaded *before*
    the RS.

    Perhaps someone can clarify things a little for me. For example, does
    the FS really load before the RS? If so, how does it become a child of
    the RS? The book says that everything but the PM is a child of the RS.

    I can think of another thing that makes the FS special: The FS and the
    PM share the job of making the POSIX interface, and they with the
    kernel manage the process table (each one has a different portion of
    the process table). That tells me that the FS, PM and the kernel are
    the three most fundamental components of MInix. But at the same time
    the RS is in charge of overseeing the FS in case it dies, right?

    Daniel.

  2. Re: File System, Reincarnation Server and Process Manager

    Daniel Carrera wrote:
    > Hello,


    > Reading through the Minix book, I learned that ever process besides
    > the Process Manager (PM) is is a child (or descendant) of the
    > Reincanration Server (RS). My question: Is this true for the File
    > Server (FS) too?


    > Reading through the book, some times it seems like the PM and the FS
    > are the most fundamental proceses in Minix 3 (after the kernel).
    > Scanning the source code, they seem to have a bit of a special role,
    > and I notice in Figure 2-31 that the FS appears to be loaded *before*
    > the RS.


    > Perhaps someone can clarify things a little for me. For example, does
    > the FS really load before the RS? If so, how does it become a child of
    > the RS? The book says that everything but the PM is a child of the RS.


    > I can think of another thing that makes the FS special: The FS and the
    > PM share the job of making the POSIX interface, and they with the
    > kernel manage the process table (each one has a different portion of
    > the process table). That tells me that the FS, PM and the kernel are
    > the three most fundamental components of MInix. But at the same time
    > the RS is in charge of overseeing the FS in case it dies, right?


    Hey Daniel,

    The RS is indeed in charge of overseeing as much as possible in case
    something dies, and though I have not tested it myself, I believe the RS
    can restart the FS. The reason the PM cannot be restarted is that the PM
    is still critical to restarting a process, so it can't help itself
    restart.

    Now for the tricky part, the parent/child relation between the FS and RS.
    This is hypothetical, but I think it is how it works in reality as well:
    the FS is loaded before the RS, but it is indeed a child of RS. How, you
    say? Remember that PM, FS and RS are all special and also fundamental
    parts of the system. Although not part of the kernel, it is known at
    compile time all three will be there so special cases can be hardcoded for
    them. Therefore it is perfectly possible that sometime during the start of
    the system, parent/child relations are shifted somewhat and that
    parenthood of all processes except PM transfer to RS. After all, these
    relationships formally consist of only a few bytes in the process table,
    changing them shouldn't be too hard.

    Remember, OS internals consist of at least 20% black magic, changing the
    rules of reality

    Regards,

    Jens

    --
    Jens de Smit
    Student Computer Science | Vrije Universiteit Amsterdam
    jfdsmit@few.vu.nl | http://www.few.vu.nl/~jfdsmit
    "[In the end, people] get furious at IT that the goddamn magic isn't working"
    -- Stewart Dean

  3. Re: File System, Reincarnation Server and Process Manager

    On Sep 1, 9:49*am, "J.F. de Smit" wrote:
    > The RS is indeed in charge of overseeing as much as possible in case
    > something dies, and though I have not tested it myself, I believe the RS
    > can restart the FS. The reason the PM cannot be restarted is that the PM
    > is still critical to restarting a process, so it can't help itself
    > restart.


    I was wondering about that, but now it all makes sense. Reviving a
    child requires fork() and exec(). PM is who provides fork() and
    exec(), so you can't revive PM.


    > Now for the tricky part, the parent/child relation between the FS and RS.*
    > This is hypothetical, but I think it is how it works in reality as well: *
    > the FS is loaded before the RS, but it is indeed a child of RS. How, you
    > say? Remember that PM, FS and RS are all special and also fundamental
    > parts of the system. Although not part of the kernel, it is known at
    > compile time all three will be there so special cases can be hardcoded for
    > them. Therefore it is perfectly possible that sometime during the start of
    > the system, parent/child relations are shifted somewhat and that
    > parenthood of all processes except PM transfer to RS. After all, these
    > relationships formally consist of only a few bytes in the process table,
    > changing them shouldn't be too hard.


    So you think that FS is started directly by the kernel and later
    becomes "adopted" by RS? That sounds reasonable.


    I know that Minix can "re-parent" processes, but the examples I knew
    are where a grand-child becomes a child, so it's a little different.
    Ex: init starts some servers but they become adopted by RS. Maybe the
    mechanism for reparenting FS is similar.


    > Remember, OS internals consist of at least 20% black magic, changing the
    > rules of reality


    :-)

    Thanks for the clarifications.

    Daniel.

  4. Re: File System, Reincarnation Server and Process Manager

    Daniel Carrera wrote:
    > So you think that FS is started directly by the kernel and later
    > becomes "adopted" by RS? That sounds reasonable.


    IIRC, all "important" processes like init, pm, fs, rs and the "kernel
    process" (system and clock task) are actually part of the boot image and
    are put into memory directly rather than started explicitly. A
    construction like this is necessary because without an active FS it is
    impossible to read the FS' program image from disk and load it in a
    regular manner. The FS and PM actually "hit the ground running", so to
    say.

    Regards,

    Jens

    --
    Jens de Smit
    Student Computer Science | Vrije Universiteit Amsterdam
    jfdsmit@few.vu.nl | http://www.few.vu.nl/~jfdsmit
    "[In the end, people] get furious at IT that the goddamn magic isn't working"
    -- Stewart Dean

  5. Re: File System, Reincarnation Server and Process Manager

    On Sep 1, 12:42*pm, "J.F. de Smit" wrote:
    > IIRC, all "important" processes like init, pm, fs, rs and the "kernel
    > process" *(system and clock task) are actually part of the boot image and
    > are put into memory directly rather than started explicitly. A
    > construction like this is necessary because without an active FS it is
    > impossible to read the FS' program image from disk and load it in a
    > regular manner. The FS and PM actually "hit the ground running", so to
    > say.


    I hadn't thought of that. I saw in the book that those components are
    in the boot image, but I didn't realize how important that is. The FS
    must be in the image because you can't read any files without the FS.
    Ditto for the disk driver.

    For reference, the boot image comonents are (in order): kernel, pm,
    fs, rs, console driver, memory driver, log driver, disk driver, init.

    I can see how all of these (except the console and logs) absolutely
    have to be in the boot image. The console are logs are just nice to
    have.

    OH!! I think I figured out why the FS is loaded before the RS. The FS
    contains part of the process table. The RS is a process. When a
    process is created the PM sends a message to the FS to update its part
    of the process table. So the sequence might be like this:

    1. PM starts.
    2. FS starts.
    3. PM sends a message to FS saying "process FS has started".
    4. FS receives message.
    5. RS starts.
    6. PM sends a message to FS saying "process RS has started".
    ....

    If the RS started first, there would be no one to receive the message
    from the PM.

    But this makes me wonder, if the FS is as important as I think, how
    could the RS restart it after it crashes? Consier:

    1. FS dies.
    2. PM sends a message to FS saying "process FS has died".
    3. But there is no FS to receive that message.

    Wikipedia says "Currently the reincarnation server, the file server,
    the process server, and the microkernel are part of the trusted
    computing base. If any of them fail, the system crashes".

    I think this makes sense:

    * FS: The argument above suggests that we can't survive a FS crash.
    * PM: You can't reincarnate without fork+exec. So you can't
    reincarnate PM.
    * RS: You can't reincarnate without a reincarnation server.

    But this makes me wonder: What's the point of making the FS a child of
    the RS if the RS can't reincarnate it? But yet, the book clearly says
    that the only server that is not a child of the RS is the PM.

    Daniel.

  6. Re: File System, Reincarnation Server and Process Manager

    Daniel Carrera wrote:
    > But this makes me wonder: What's the point of making the FS a child of
    > the RS if the RS can't reincarnate it? But yet, the book clearly says
    > that the only server that is not a child of the RS is the PM.


    Maybe there are plans to make at least the FS reincarnable (is that a
    word?) by having a backup copy of the FS image somewhere (perhaps in the
    RS' memory space?) and the current implementation is already prepared for
    that? Otherwise I wouldn't really know, maybe "just because" is the
    correct answer...

    Regards,

    Jens

    --
    Jens de Smit
    Student Computer Science | Vrije Universiteit Amsterdam
    jfdsmit@few.vu.nl | http://www.few.vu.nl/~jfdsmit
    "[In the end, people] get furious at IT that the goddamn magic isn't working"
    -- Stewart Dean

  7. Re: File System, Reincarnation Server and Process Manager

    J.F. de Smit escreveu:

    > Daniel Carrera wrote:
    >> So you think that FS is started directly by the kernel and later
    >> becomes "adopted" by RS? That sounds reasonable.

    >
    > IIRC, all "important" processes like init, pm, fs, rs and the "kernel
    > process" (system and clock task) are actually part of the boot image and
    > are put into memory directly rather than started explicitly.


    From what I have read, there's no clock thread any more.

    Also, some stuff you mentioned is (or if it isn't, at least could be)
    started normaly. For example, by reading the FS code (pre-VFS):
    ---------------------------------------------
    /* Get some boot environment variables. */
    root_dev = igetenv("rootdev", 0);
    ---------------------------------------------
    If I understand correctly, the FS mounts the initial root partition itself
    (i.e. without being instructed by init). The above code reveals that FS is
    informed of the boot device during the initialization phase. Of course,
    some other low level process needs to be started in order to mount the
    first file system. I guess it's the RAM disk driver. (Note that I'm just
    guessing)

    This way, the init could be avoided to be included in the boot image.
    Furthermore, I've read in Minix VFS Design and Implementation that VFS is
    re-executed at some point during the boot phase, this time from that RAM
    disk.

    > A construction like this is necessary because without an active
    > FS it is impossible to read the FS' program image from disk and
    > load it in a regular manner. The FS and PM actually "hit the
    > ground running", so to say.


    This is strange, as PM was one of the subsystems that always ran in
    userspace on Minix.

    --
    João Jerónimo

    "Computer are composed of software, hardware, and other stuff terminated
    in "ware", like firmware, tupperware, (...)" - by JJ.

  8. Re: File System, Reincarnation Server and Process Manager

    J.F. de Smit escreveu:

    > Maybe there are plans to make at least the FS reincarnable (is that a
    > word?) by having a backup copy of the FS image somewhere (perhaps in the
    > RS' memory space?) and the current implementation is already prepared for
    > that? Otherwise I wouldn't really know, maybe "just because" is the
    > correct answer...


    I've been thinking about that (partly because the system does not seem to
    survive a simple "kill -KILL" issued to a process that manages a mounted
    partition). My view is that vital processes (like PM and VFS) could start
    their life by forking themselves, and thus maintaining a backup copy.
    Then, if one of them dies, the RS could send their backup instance a special
    message asking for them the refork and make their best at repairing
    themselves.
    Of course, there is a HUGE problem: both PM and VFS maintain tons of
    process-related vital system information, and restoring their state from
    backup copies wouldn't be an easy task at all.

    --
    João Jerónimo

    "Computer are composed of software, hardware, and other stuff terminated
    in "ware", like firmware, tupperware, (...)" - by JJ.

  9. Re: File System, Reincarnation Server and Process Manager

    J.F. de Smit escreveu:

    > The RS is indeed in charge of overseeing as much as possible in case
    > something dies, and though I have not tested it myself, I believe the RS
    > can restart the FS. The reason the PM cannot be restarted is that the PM
    > is still critical to restarting a process, so it can't help itself
    > restart.


    The RS is itself responsible for managing drivers & servers. It's more or
    less analogous to the job init has in System V; but for the vital
    processes, instead of classical services.
    -----------------------------------------------------------------------
    In contrast, init executes the /etc/rc script that causes it to issue
    commands to the reincarnation server to start the drivers and servers not
    present in the boot image.
    -----------------------------------------------------------------------

    Just as init kills and loads services when someone asks to switch the
    runlevel (by using telinit), RS loads and kill services when asked to.

    --
    João Jerónimo

    "Computer are composed of software, hardware, and other stuff terminated
    in "ware", like firmware, tupperware, (...)" - by JJ.

  10. Re: File System, Reincarnation Server and Process Manager

    Jo??o Jer??nimo wrote:
    > J.F. de Smit escreveu:


    > From what I have read, there's no clock thread any more.


    Minix never had threads, so you can leave the "any more".

    >> A construction like this is necessary because without an active
    >> FS it is impossible to read the FS' program image from disk and
    >> load it in a regular manner. The FS and PM actually "hit the
    >> ground running", so to say.


    > This is strange, as PM was one of the subsystems that always ran in
    > userspace on Minix.


    Can you please explain where I suggested it doesn't run in userspace?

    Regards,

    Jens

    --
    Jens de Smit
    Student Computer Science | Vrije Universiteit Amsterdam
    jfdsmit@few.vu.nl | http://www.few.vu.nl/~jfdsmit
    "[In the end, people] get furious at IT that the goddamn magic isn't working"
    -- Stewart Dean

  11. Re: File System, Reincarnation Server and Process Manager

    J.F. de Smit escreveu:

    >> From what I have read, there's no clock thread any more.

    >
    > Minix never had threads, so you can leave the "any more".


    I meant process. (And it ran in the kernel's address space, afaik)

    >>> A construction like this is necessary because without an active
    >>> FS it is impossible to read the FS' program image from disk and
    >>> load it in a regular manner. The FS and PM actually "hit the
    >>> ground running", so to say.

    >> This is strange, as PM was one of the subsystems that always ran in
    >> userspace on Minix.

    >
    > Can you please explain where I suggested it doesn't run in userspace?


    Nowhere.

    --
    João Jerónimo

    "Computer are composed of software, hardware, and other stuff terminated
    in "ware", like firmware, tupperware, (...)" - by JJ.

+ Reply to Thread