Roman, good day.

Sat, Sep 27, 2008 at 08:18:08PM +0400, Roman Kurakin wrote:
> Have you also posted this to ports@?

No, forgot to do it. CC'ing ports@

Thanks!

The original posting to hackers@ goes below. It will be double-posted
to the bug-followup@ -- sorry for this.

> Eygene Ryabinkin wrote:
> > Good day.
> >
> > A while ago I had created the new utility that serves as VuXML
> > filter for the installed packages:
> > http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/126853
> >
> > My primary intention was to speed up the process of auditing the
> > vulnerable ports: I needed to run portaudit checks with Nagios and to
> > avoid large timeouts.
> >
> > The new utility is called pkg_audit and it serves as a simple text
> > filter: on input it takes the full VuXML feed and on output it puts
> > VuXML entries that matches ports that are installed in the system with
> > port version specification substituted with the actual port versions.
> >
> > No harm is done to the actual poartudit -- if pkg_audit is missing, old
> > code path is activated.
> >
> > If someone is interested and will be able to test -- I am all ears.

Additional clarifications inspired by the off-line talk with rik@:
I could take another route and add this functionality to the pkg_info.
I took another approach for the following reasons.

1. pkg_info's option list is already quite big -- around 32 options
and switches.

2. It is easier to test for the presence of the new tool (pkg_audit)
and use it, instead of checking the support for the new option in
pkg_info.

3. I see no options in pkg_info that can be naturally extended to
absorbe the new functionality. The closest is '-E', but pkg_audit
needs to read VuXML entries, choose ones that are present in the system
and output the found VuXML entries with version templates substituted
with the real entries, so pkg_audit is filter-like utility. In my
opinion, such extension of pkg_info's "-E" will be very unnatural.

4. I feel that it is Unix-way to do the things: create small utilities
that do their (small) job in a proper fashion. Moreover, since the
majority of a code sits in the pkg_install's library, there is a very
slight code duplication, if any.

Thanks for you time.
--
Eygene
_ ___ _.--. #
\`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard
/ ' ` , __.--' # to read the on-line manual
)/' _/ \ `-_, / # while single-stepping the kernel.
`-'" `"\_ ,_.-;_.-\_ ', fsc/as #
_.-'_./ {_.' ; / # -- FreeBSD Developers handbook
{_.-``-' {_/ #

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (FreeBSD)

iEYEARECAAYFAkjfUx4ACgkQthUKNsbL7YhBbwCfSA2OhrF4uP 6vfYW/eQ2kQ7/p
LbgAmwTYjAJor/5W5Tc5lzeVW3RQXgQx
=xsRg
-----END PGP SIGNATURE-----

Miroslav, good day.

Mon, Oct 06, 2008 at 12:41:05AM +0200, Miroslav Lachman wrote:
> I am busy these days, but it is nice to read about your progress. I hope
> I will get some time to test all of these large patches in a few days
> and I will report back my experiences!

Fine, thank you! I am re-CC'ing bug-followup@ to track this letter,
since it contains some useful information that should go into GNATS.

> One note before tests... do -n flag always download new INDEX file, or
> is it possible to use one already existing in /usr/ports?

Currently, it is downloads bzipped INDEX file to /var/db/portaudit every
time, but it uses mirror mode, so if remote file hadn't changed at all,
all network expences are just the HTTP's HEAD request and reply.

I can add another variable to the portaudit to force the usage of the
existing INDEX file, if it is needed. By the way, how are you keeping
your INDEX file up to date (your proposed usage of 'pkg_version -I'
implies that you're always rely on it)? I am just curious -- my INDEX
files are almost always stay unupdated, even if I am using portupgrade.

And there can be another way if one keeps ports tree updated: utility
can use 'make' to determine the version that is currently available on
the examined host.

But downloading the INDEX file from the central server seemed to be the
best way, since it almost always gives one the latest port versions, so
I had implemented this in a first place.

Don't know, however, how the badly the load to the central HTTP server
will be raised. I am using just two first fields from the INDEX file,
so I can use such a stripped file. For me, the reduction was about
6x: SIZE(INDEX-7.bz2) = 1126189, SIZE(INDEX-7.stripped.bz2) = 184345.

I am CC'ing the portmgr team. Guys, could you quickly glance over these
patches and determine if they are useful to the project in large? If
yes, then may be such a stripped INDEX can be created on the FreeBSD
servers (via cut -f1-2 -d'|' INDEX-N)?

Thanks!
--
Eygene
_ ___ _.--. #
\`.|\..----...-'` `-._.-'_.-'` # Remember that it is hard
/ ' ` , __.--' # to read the on-line manual
)/' _/ \ `-_, / # while single-stepping the kernel.
`-'" `"\_ ,_.-;_.-\_ ', fsc/as #
_.-'_./ {_.' ; / # -- FreeBSD Developers handbook
{_.-``-' {_/ #

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (FreeBSD)

iEYEARECAAYFAkjpoNkACgkQthUKNsbL7YhirACeIewIwCM0wr/UBHnMp7hieJya
eUoAoIAwCdHLh38sZwl+bvw65cB9OvgW
=ANyt
-----END PGP SIGNATURE-----

Hello,

On Monday 06 October 2008 07:23:37 Eygene Ryabinkin wrote:

> But downloading the INDEX file from the central server seemed to be the
> best way, since it almost always gives one the latest port versions, so
> I had implemented this in a first place.

I've been following this, but I don't agree that (port|pkg_)audit should do
this, from the very perspective you're writing this program from:

On Sunday 28 September 2008 11:49:18 Eygene Ryabinkin wrote:
> 4. I feel that it is Unix-way to do the things: create small utilities
> that do their (small) job in a proper fashion.

Instead, it can provide installed-pkgnamepkgorigin output. Then,
any utility can check whether a new version is available, using what ever
source it finds relevant.
For example, it is completely irrelevant if a new version is available on the
FreeBSD servers, when your machine uses a buildserver in a local network. For
those machines it's relevant whether their build server has a new version and
one can automatically upgrade if one so desires.
Similarly, if your /usr/ports is ahead of the FreeBSD's INDEX.bz2, you're
again reporting false information.

It's also quite trivial to provide this availibility information in a daily
security script, for the "majority of cases" and it's better to have tunables
like _use_remote_portindex, _use_portsdir=/bigdisk/usr/ports in a script.
--
Mel
_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/lis...reebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"