DICOM File Meta Information: VR=UN - DICOM

This is a discussion on DICOM File Meta Information: VR=UN - DICOM ; Hi there, After reading PS 3.10 - 2008, 7.1 DICOM FILE META INFORMATION, I believe that a DICOM file is valid even if the elements in the File Meta Information are declared as UN instead of there respective VR. I ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: DICOM File Meta Information: VR=UN

  1. DICOM File Meta Information: VR=UN

    Hi there,

    After reading PS 3.10 - 2008, 7.1 DICOM FILE META INFORMATION, I
    believe that a DICOM file is valid even if the elements in the File
    Meta Information are declared as UN instead of there respective VR.
    I know the case is hairy, but when converting a Little Endian
    Implicit file meta information header (typically from GE), I do not
    wish to load the data dictionary in which case I default to using
    VR=UN.

    Thanks
    -Mathieu

  2. Re: DICOM File Meta Information: VR=UN

    > After reading PS 3.10 - 2008, 7.1 DICOM FILE META INFORMATION, I
    > believe that a DICOM file is valid even if the elements in the File
    > Meta Information are declared as UN instead of there respective VR.
    > I know the case is hairy, but when converting a Little Endian
    > Implicit file meta information header (typically from GE), I do not
    > wish to load the data dictionary in which case I default to using
    > VR=UN.


    Certainly a gray area, but I believe that encoding meta-headers as UN
    will break many toolkits (certainly ours, i.e. DCMTK, in the default
    configuration) and is not a good idea.
    The only reason to ever create UN elements is when converting from
    Implicit VR to Explicit VR and reading an attribute that is unknown
    to the application. However, no metadata attributes have been defined
    since part 10 was written, so there is no reason why an application
    writing a part 10 metaheader should ever encounter an unknown attribute.
    Also, "conversion from Implicit VR metaheader" is a use-case that does
    not exist in DICOM since a metaheader cannot ever exist in Implicit VR
    format (of course we all know that not all developers read the Final Text
    version of the specs, but correct behaviour in face of faulty data is
    something that is an area where DICOM does not define any rules).

    In summary, not wishing to load a data dictionary (which in this case
    would just need to contain about 10 attributes) is a rather lame excuse
    and the converted files will probably cause more problems than the faulty
    orginal ones.

    Regards,
    Marco Eichelberg

  3. Re: DICOM File Meta Information: VR=UN

    On Mar 4, 9:23 am, Marco Eichelberg
    wrote:
    > > After reading PS 3.10 - 2008, 7.1 DICOM FILE META INFORMATION, I
    > > believe that a DICOM file is valid even if the elements in the File
    > > Meta Information are declared as UN instead of there respective VR.
    > > I know the case is hairy, but when converting a Little Endian
    > > Implicit file meta information header (typically from GE), I do not
    > > wish to load the data dictionary in which case I default to using
    > > VR=UN.

    >
    > Certainly a gray area, but I believe that encoding meta-headers as UN
    > will break many toolkits (certainly ours, i.e. DCMTK, in the default
    > configuration) and is not a good idea.
    > The only reason to ever create UN elements is when converting from
    > Implicit VR to Explicit VR and reading an attribute that is unknown
    > to the application. However, no metadata attributes have been defined
    > since part 10 was written, so there is no reason why an application
    > writing a part 10 metaheader should ever encounter an unknown attribute.
    > Also, "conversion from Implicit VR metaheader" is a use-case that does
    > not exist in DICOM since a metaheader cannot ever exist in Implicit VR
    > format (of course we all know that not all developers read the Final Text
    > version of the specs, but correct behaviour in face of faulty data is
    > something that is an area where DICOM does not define any rules).
    >
    > In summary, not wishing to load a data dictionary (which in this case
    > would just need to contain about 10 attributes) is a rather lame excuse
    > and the converted files will probably cause more problems than the faulty
    > orginal ones.


    Depends what you call 'lame'. I call it a lame excuse not to go and
    fix a toolkit for 'gray area' of the standard :-P
    Anyway -at least for me- it completely breaks the separation in
    between Part 3.5 and Part 3.6/Part 3.3. No data dictionary should be
    needed to read a DICOM file (parsing is a different operation than
    interpreting).
    Call it a design issue if you want, but having the data dictionary
    pulls in for example dependency to an XML library. And even in C++
    constructing a hash table (std::map) is insanely slow.

    And for reference, the following do not seems to have any particular
    issues with such files:
    - dicom3tools
    - TomoVision
    - irfanview
    - dicomworks
    - Efilm 153

    -Mathieu
    Ref:
    http://gdcm.sf.net/PICKER-16-MONO2-N...3_Preamble.dcm

  4. Re: DICOM File Meta Information: VR=UN

    Hi Mathieu

    This is a really bad idea.

    Just in case it is not obvious, I will put in a CP to explicitly
    state that these attributes shall never use the UN VR.

    David

    Mathieu Malaterre wrote:
    > Hi there,
    >
    > After reading PS 3.10 - 2008, 7.1 DICOM FILE META INFORMATION, I
    > believe that a DICOM file is valid even if the elements in the File
    > Meta Information are declared as UN instead of there respective VR.
    > I know the case is hairy, but when converting a Little Endian
    > Implicit file meta information header (typically from GE), I do not
    > wish to load the data dictionary in which case I default to using
    > VR=UN.
    >
    > Thanks
    > -Mathieu


  5. Re: DICOM File Meta Information: VR=UN

    Mathieu Malaterre wrote:
    > Depends what you call 'lame'. I call it a lame excuse not to go and
    > fix a toolkit for 'gray area' of the standard :-P


    Point taken :-)

    That said, most of the "gray areas" in DICOM only turned out to be
    gray areas when somebody came up with an interpretation I had never
    thought to be ambiguous - like this example.

    With DCMTK 3.5.4, dcmdump will refuse to even read the file,
    while dcmdump --convert-un will work correctly.

    The problem is that by default DCMTK makes use of the meta-header
    group length attribute, and the parser refuses to interpret that as UL
    if encoded in UN in the dataset. One could certainly implement a workaround
    (such as enabling --convert-un by default when reading a meta-header),
    but that is not present in any DCMTK release up to now.

    > Anyway -at least for me- it completely breaks the separation in
    > between Part 3.5 and Part 3.6/Part 3.3. No data dictionary should be
    > needed to read a DICOM file (parsing is a different operation than
    > interpreting).


    That is a good question. Without a dictionary, you cannot identify
    sequences (with explicit length), so your parser cannot create an
    in-memory tree representing all elements. You need to treat sequences
    as "black boxes" and parse them at a later point when their VR is known,
    which again breaks the separation. In summary, I don't think that the
    DICOM encoding really permits a "clean" parsing without a data dictionary.
    If Implicit VR had gone away in 1993, this might be easier, but with
    Implicit VR you simply need a-priori knowledge if you want your parser
    to produce anything useful.

    > Call it a design issue if you want, but having the data dictionary
    > pulls in for example dependency to an XML library. And even in C++
    > constructing a hash table (std::map) is insanely slow.


    That might depend on the hash table implementation - std::map might
    not be the optimal choice here. I agree that creating the dictionary
    is relatively expensive (in the order of a few milliseconds on modern hardware),
    but not to the degree that it really hurts.

    One compromise would be to create a "skeleton dictionary" that is always
    available and built in. DCMTK actually does this, although the skeleton
    only contains (xxxx,0000), (fffe,e000), (fffe,e00d) and (fffe,e0dd),
    i.e. group length and the special-purpose tags for items and delimiters.
    One could imagine extending that with the few group 0000 and 0002 tags
    which would permit a reliable reading at least of metaheaders and network
    messages. Such a dictionary with, say, 20 tags should not be expensive
    to construct.

    Regards,
    Marco Eichelberg

  6. Re: DICOM File Meta Information: VR=UN

    On Mar 4, 1:25 pm, Marco Eichelberg
    wrote:
    > Mathieu Malaterre wrote:
    > > Depends what you call 'lame'. I call it a lame excuse not to go and
    > > fix a toolkit for 'gray area' of the standard :-P

    >
    > Point taken :-)
    >
    > That said, most of the "gray areas" in DICOM only turned out to be
    > gray areas when somebody came up with an interpretation I had never
    > thought to be ambiguous - like this example.
    >
    > With DCMTK 3.5.4, dcmdump will refuse to even read the file,
    > while dcmdump --convert-un will work correctly.


    Cool did not realize that ! My biggest issue was that I am using
    dcmdump to verify the output of gdcm is always valid. So at least I
    have a temporary solution.

    > The problem is that by default DCMTK makes use of the meta-header
    > group length attribute, and the parser refuses to interpret that as UL
    > if encoded in UN in the dataset. One could certainly implement a workaround
    > (such as enabling --convert-un by default when reading a meta-header),
    > but that is not present in any DCMTK release up to now.
    >
    > > Anyway -at least for me- it completely breaks the separation in
    > > between Part 3.5 and Part 3.6/Part 3.3. No data dictionary should be
    > > needed to read a DICOM file (parsing is a different operation than
    > > interpreting).

    >
    > That is a good question. Without a dictionary, you cannot identify
    > sequences (with explicit length), so your parser cannot create an
    > in-memory tree representing all elements. You need to treat sequences
    > as "black boxes" and parse them at a later point when their VR is known,
    > which again breaks the separation. In summary, I don't think that the
    > DICOM encoding really permits a "clean" parsing without a data dictionary.
    > If Implicit VR had gone away in 1993, this might be easier, but with
    > Implicit VR you simply need a-priori knowledge if you want your parser
    > to produce anything useful.


    If you get a chance, please try gdcm2 (from svn on sf.net). The Parser
    is not (physically) link to the dictionary and so far I did not find a
    case where I could not recognize a SQ in implicit dataset. The whole
    idea is to look for item start/end in the binary blob. Of course one
    could imagine forging an image that exactly start with an item start
    end finish with an item end... But I like it this way, it allowed me
    to find private SQ without prior knowledge of private vendor data
    elements dictionary.

    > > Call it a design issue if you want, but having the data dictionary
    > > pulls in for example dependency to an XML library. And even in C++
    > > constructing a hash table (std::map) is insanely slow.

    >
    > That might depend on the hash table implementation - std::map might
    > not be the optimal choice here. I agree that creating the dictionary
    > is relatively expensive (in the order of a few milliseconds on modern hardware),
    > but not to the degree that it really hurts.


    People south of France likes to exagerate a little...just a little

    > One compromise would be to create a "skeleton dictionary" that is always
    > available and built in. DCMTK actually does this, although the skeleton
    > only contains (xxxx,0000), (fffe,e000), (fffe,e00d) and (fffe,e0dd),
    > i.e. group length and the special-purpose tags for items and delimiters.
    > One could imagine extending that with the few group 0000 and 0002 tags
    > which would permit a reliable reading at least of metaheaders and network
    > messages. Such a dictionary with, say, 20 tags should not be expensive
    > to construct.


    Correct. I was more concerned with the maintaining, but since meta
    header has not seen any new elements since its creation I believe this
    is safe.

    In summary, because of the coming CP (thanks David), I'll go ahead and
    fix GDCM 2.x

    Sorry for the noise,
    -Mathieu

  7. Re: DICOM File Meta Information: VR=UN

    This is not really an 'idea' but mainly because my implementation of
    File Meta Header derive from a DataSet class, which -by definition-
    uses VR=UN when the VR is unknown. I'll make sure to have a separate
    implementation for Meta Header.

    thx
    -Mathieu

    On Mar 4, 12:50 pm, David Clunie wrote:
    > Hi Mathieu
    >
    > This is a really bad idea.
    >
    > Just in case it is not obvious, I will put in a CP to explicitly
    > state that these attributes shall never use the UN VR.
    >
    > David
    >
    > Mathieu Malaterre wrote:
    > > Hi there,

    >
    > > After reading PS 3.10 - 2008, 7.1 DICOM FILE META INFORMATION, I
    > > believe that a DICOM file is valid even if the elements in the File
    > > Meta Information are declared as UN instead of there respective VR.
    > > I know the case is hairy, but when converting a Little Endian
    > > Implicit file meta information header (typically from GE), I do not
    > > wish to load the data dictionary in which case I default to using
    > > VR=UN.

    >
    > > Thanks
    > > -Mathieu



+ Reply to Thread