Hello,

on a newly installed and configured system I try to run portaudit.

web1# [~] portaudit -Fad
auditfile.tbz 100% of 49 kB 930 kBps
portaudit: Database too old.
Old database restored.
portaudit: Download failed.
web1# [~] portaudit -V
portaudit version 0.5.12
web1# [~] pkg_info -aI | grep portaudit
portaudit-0.5.12 Checks installed ports against a list of security vulnerabi
web1# [~] uname -a
FreeBSD web1.xx.xx.x 6.3-RELEASE-p2 FreeBSD 6.3-RELEASE-p2 #0: Fri Jun 13 09:18:43
CEST 2008 root@xxx:/usr/obj/usr/src/sys/SMP i386
web1# [~] date
Wed Aug 20 21:11:47 CEST 2008

I tried on systems with portaudit-0.5.11 and it keeps
working fine there. Does anybody have a suggestion how to
get portaudit working?

Regards

Christoph Weber-Fahr

Christoph Weber-Fahr wrote:
> Hello,
>
> on a newly installed and configured system I try to run portaudit.
>
> web1# [~] portaudit -Fad
> auditfile.tbz 100% of 49 kB 930 kBps
> portaudit: Database too old.
> Old database restored.
I encounter the same problem if I use a proxy (squid 3.0)
eg:
portaudit_fetch_env="HTTP_PROXY=http://proxy.restart.bel:3128/
in /usr/local/etc/portaudit.conf

or HTTP_PROXY is defined in the environment (/etc/profile for example)

Henri

> portaudit: Download failed.
> web1# [~] portaudit -V
> portaudit version 0.5.12
> web1# [~] pkg_info -aI | grep portaudit
> portaudit-0.5.12 Checks installed ports against a list of security
> vulnerabi
> web1# [~] uname -a
> FreeBSD web1.xx.xx.x 6.3-RELEASE-p2 FreeBSD 6.3-RELEASE-p2 #0: Fri Jun
> 13 09:18:43
> CEST 2008 root@xxx:/usr/obj/usr/src/sys/SMP i386
> web1# [~] date
> Wed Aug 20 21:11:47 CEST 2008
>
> I tried on systems with portaudit-0.5.11 and it keeps
> working fine there. Does anybody have a suggestion how to
> get portaudit working?
>
> Regards
>
> Christoph Weber-Fahr
>

Henri Hennebert wrote:
> Christoph Weber-Fahr wrote:

>> web1# [~] portaudit -Fad
>> auditfile.tbz 100% of 49 kB 930 kBps
>> portaudit: Database too old.
>> Old database restored.
>
> I encounter the same problem if I use a proxy (squid 3.0)
> eg:
> portaudit_fetch_env="HTTP_PROXY=http://proxy.restart.bel:3128/
> in /usr/local/etc/portaudit.conf
>
> or HTTP_PROXY is defined in the environment (/etc/profile for example)

Yep. My system also uses a proxy.

But since it uses fetch in both cases, that shouldn't make a difference,
should it?

Regards

Christoph Weber-Fahr

Christoph Weber-Fahr wrote:
> Henri Hennebert wrote:
>> Christoph Weber-Fahr wrote:
>
>>> web1# [~] portaudit -Fad
>>> auditfile.tbz 100% of 49 kB 930 kBps
>>> portaudit: Database too old.
>>> Old database restored.
>>
>> I encounter the same problem if I use a proxy (squid 3.0)
>> eg:
>> portaudit_fetch_env="HTTP_PROXY=http://proxy.restart.bel:3128/
>> in /usr/local/etc/portaudit.conf
>>
>> or HTTP_PROXY is defined in the environment (/etc/profile for example)
>
> Yep. My system also uses a proxy.
>
> But since it uses fetch in both cases, that shouldn't make a difference,
> should it?
Obviously, the proxy don't querry the last modification time of the file
and deliver a local copy.

I make some test and was surprise by this behavior. IIRC this problem
appear for me after an upgrade to squid 3.0.

Henri
>
> Regards
>
> Christoph Weber-Fahr

Hello,

Henri Hennebert wrote:
> Christoph Weber-Fahr wrote:
>> Henri Hennebert wrote:
>>> Christoph Weber-Fahr wrote:
>>>> web1# [~] portaudit -Fad
>>>> auditfile.tbz 100% of 49 kB 930 kBps
>>>> portaudit: Database too old.
>>>> Old database restored.

> Obviously, the proxy don't querry the last modification time of the file
> and deliver a local copy.

Yep, looks like that. In my case it is a squid 2.6.STABLE19,
wich is chained through another squid.

Apparently it had a broken copy, since it always delivered it
with the most recent date.

Fortunatley it was my squid, so I could just reset its cache. Now it
behaves correctly, consistently delivering the current file witha
correct mtime.

But fetch seems to be guilty, too. See this transcript (I added -vv
in the fetch command in portaudit.conf):

> portaudit -Fad
> scheme: [http]
> user: []
> password: []
> host: [www.FreeBSD.org]
> port: [0]
> document: [/ports/auditfile.tbz]
> scheme: [http]
> user: []
> password: []
> host: [127.0.0.1]
> port: [8000]
> document: [/]
> ---> 127.0.0.1:8000
> looking up 127.0.0.1
> connecting to 127.0.0.1:8000

That one is ssh-portforwarded to the proxy:

> requesting http://www.FreeBSD.org/ports/auditfile.tbz
>>>> GET http://www.FreeBSD.org/ports/auditfile.tbz HTTP/1.1
>>>> Host: www.FreeBSD.org
>>>> User-Agent: fetch libfetch/2.0
>>>> Connection: close
>>>>
> <<< HTTP/1.0 200 OK
> <<< Content-Type: application/x-bzip-compressed-tar
> <<< Accept-Ranges: bytes
> <<< ETag: "-1789851444"
> <<< Last-Modified: Tue, 24 Jun 2008 12:40:02 GMT
> last modified: [2008-06-24 12:40:02]

Here, apparently, fetch could have learnt the right time - but didn't.
Yeah, its badly outdated - no idea why squid doesn't attempt an update
of that file.

> <<< Content-Length: 50554
> content length: [50554]
> <<< Date: Tue, 24 Jun 2008 12:43:21 GMT
> <<< Server: httpd/1.4.x LaHonda
> <<< X-Cache: MISS from proxy4
> <<< Age: 200385
> <<< X-Cache: HIT from mybrokenproxy.xx.xx
> <<< Via: 1.0 proxy4:8000 (squid), 1.0 mybrokenproxy.xx.xx:8000 (squid/2.6.STABLE19)
> <<< Proxy-Connection: close
> <<<
> offset 0, length -1, size -1, clength 50554
> local size / mtime: 50554 / 1219349222
> remote size / mtime: 50554 / 1214311202

See the mtime value here. For reasons unknown to me fetch decides to set a
different local mtime.
> auditfile.tbz 100% of 49 kB 308 kBps
> portaudit: Database too old.
> Old database restored.
> portaudit: Download failed.

And I just reproduced that after the cleaned cache. Apparently with chained
proxies squid can't update the thing properly and keeps an old copy around.

Looks Fishy.
Maybe someone with more http and caching know how can add details here?


Regards

Christoph Weber-Fahr

Christoph Weber-Fahr wrote:
> Hello,
>
> Henri Hennebert wrote:
>> Christoph Weber-Fahr wrote:
>>> Henri Hennebert wrote:
>>>> Christoph Weber-Fahr wrote:
>>>>> web1# [~] portaudit -Fad
>>>>> auditfile.tbz 100% of 49 kB 930
>>>>> kBps
>>>>> portaudit: Database too old.
>>>>> Old database restored.
>
>> Obviously, the proxy don't querry the last modification time of the
>> file and deliver a local copy.
>
> Yep, looks like that. In my case it is a squid 2.6.STABLE19,
> wich is chained through another squid.
>
> Apparently it had a broken copy, since it always delivered it
> with the most recent date.
>
> Fortunatley it was my squid, so I could just reset its cache. Now it
> behaves correctly, consistently delivering the current file witha
> correct mtime.
>
> But fetch seems to be guilty, too. See this transcript (I added -vv
> in the fetch command in portaudit.conf):
>
>> portaudit -Fad
>> scheme: [http]
>> user: []
>> password: []
>> host: [www.FreeBSD.org]
>> port: [0]
>> document: [/ports/auditfile.tbz]
>> scheme: [http]
>> user: []
>> password: []
>> host: [127.0.0.1]
>> port: [8000]
>> document: [/]
>> ---> 127.0.0.1:8000
>> looking up 127.0.0.1
>> connecting to 127.0.0.1:8000
>
> That one is ssh-portforwarded to the proxy:
>
>> requesting http://www.FreeBSD.org/ports/auditfile.tbz
>>>>> GET http://www.FreeBSD.org/ports/auditfile.tbz HTTP/1.1
>>>>> Host: www.FreeBSD.org
>>>>> User-Agent: fetch libfetch/2.0
>>>>> Connection: close
>>>>>
>> <<< HTTP/1.0 200 OK
>> <<< Content-Type: application/x-bzip-compressed-tar
>> <<< Accept-Ranges: bytes
>> <<< ETag: "-1789851444"
>> <<< Last-Modified: Tue, 24 Jun 2008 12:40:02 GMT
>> last modified: [2008-06-24 12:40:02]
>
> Here, apparently, fetch could have learnt the right time - but didn't.
> Yeah, its badly outdated - no idea why squid doesn't attempt an update
> of that file.
>
>> <<< Content-Length: 50554
>> content length: [50554]
>> <<< Date: Tue, 24 Jun 2008 12:43:21 GMT
>> <<< Server: httpd/1.4.x LaHonda
>> <<< X-Cache: MISS from proxy4
>> <<< Age: 200385
>> <<< X-Cache: HIT from mybrokenproxy.xx.xx
>> <<< Via: 1.0 proxy4:8000 (squid), 1.0 mybrokenproxy.xx.xx:8000
>> (squid/2.6.STABLE19)
>> <<< Proxy-Connection: close
>> <<<
>> offset 0, length -1, size -1, clength 50554
>> local size / mtime: 50554 / 1219349222
>> remote size / mtime: 50554 / 1214311202
>
> See the mtime value here. For reasons unknown to me fetch decides to set a
> different local mtime.

I test and find that the file already exist locally and fetch simply
display it's mtime (which is `date -u -r 1219349222` = Thu Aug 21
20:07:02 UTC 2008) and then replace it with the stale copy comming from
the proxy.

You can reproduce it like this:

cd /tmp
fetch -v -v http://www.freebsd.org/ports/auditfile.tbz
ls -l auditfile.tbz
touch auditfile.tbz
fetch -v -v http://www.freebsd.org/ports/auditfile.tbz
ls -l auditfile.tbz

>> auditfile.tbz 100% of 49 kB 308 kBps
>> portaudit: Database too old.
>> Old database restored.
>> portaudit: Download failed.
>
> And I just reproduced that after the cleaned cache. Apparently with chained
> proxies squid can't update the thing properly and keeps an old copy around.

try to purge the entry with:

squidclient -u admin_user -w admin_password -m purge
http://www.FreeBSD.org/ports/auditfile.tbz

on mybrokenproxy.xx.xx

Henri
>
> Looks Fishy.
> Maybe someone with more http and caching know how can add details here?
>
>
> Regards
>
> Christoph Weber-Fahr