On Tue, Jul 03, 2007, Steve Langasek wrote:

> > So, an idea: what about checking "make -f /dev/null blah 2>/dev/null" first,
> > for some portability?
>
> What 'blah' are you planning to use that's guaranteed to not have broken
> side-effects in some cases on Debian packages?

How about: "blah$((uptime;ps aux) | sha1sum | cut -f1 -d-)"

Cheers,
--
Sam.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

On Tue, Jul 03, 2007 at 10:01:47AM -0700, Steve Langasek wrote:
> On Tue, Jul 03, 2007 at 09:54:03AM +0200, Adam Borowski wrote:
> > So, an idea: what about checking "make -f /dev/null blah 2>/dev/null" first,
> > for some portability?
>
> What 'blah' are you planning to use that's guaranteed to not have broken
> side-effects in some cases on Debian packages?

Any 'blah' which is not found in /dev/null, I think.
I'm not aware of any local files which can break 'make -f'.

--
1KB // Microsoft corollary to Hanlon's razor:
// Never attribute to stupidity what can be
// adequately explained by malice.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

* Russ Allbery (rra@debian.org) [070703 20:04]:
> Ian Jackson writes:
> > Steve Langasek writes ("Re: Bug#229357: Can we require build-arch/indep targets for lenny?"):
> >> Attached is a patch to dpkg which implements a check for a 'build-arch'
> >> target using 'make -f debian/rules -qn build-arch'.
>
> > Why are we so resistant to the new debian/control field ? That
> > doesn't require any of this messing about with make.
>
> Agreed. If I had my way, we'd add a Homepage debian/control field as
> well; instead we have an ugly workaround hack due to similar resistance.

I think we can still fix this mess.


Cheers,
Andi
--
http://home.arcor.de/andreas-barth/


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

On Fri, Jun 29, 2007 at 11:11:04PM +0200, Bill Allombert wrote:
> One of the issue is that tools like sbuild and pbuilder which want to
> take advantage of the Build-Depends-Indep split needs to know whether
> dpkg-buildpackage will call debian/rules build or build-arch.

It needs to know no such thing. It just needs to know that if it runs
"dpkg-buildpackage -b", only .debs will be generated, and if it runs
"dpkg-buildpackage -B", all debs apart from the _all.debs will be
generated. How exactly this happens is of no concern to sbuild.

--
Home is where you have to wash the dishes.
-- #debian-devel, Freenode, 2004-09-22


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

On Mon, Jul 02, 2007 at 03:28:05PM +0200, Simon Richter wrote:
>> Running debian-rules can always have side effects and can actively
>> rely on them so a "--has-target" can not be implemented cleanly in
>> make.
>
> I am proposing hooking into the logic that ultimately decides that
> there is no such target in the Makefile and goes on to print "Don't
> know how to make 'foo'. Stop.".

$(shell ls temp-target-* && rm temp-target-*):

Yes, that's broken, but there are your side effects, and you'll have to
run this code if you want to make your --has-target work.

--
Home is where you have to wash the dishes.
-- #debian-devel, Freenode, 2004-09-22


--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Hi,

Ian Jackson wrote:

> We want the package to _declare_ whether it supports this new target.

Ideally, we'd want all packages to support the new target, and then turn
that into policy, otherwise we'll end up supporting both for a very long
time.

> The proposed -Options field will actually be very useful for any
> other enhancements we make to the source package format.

I'm not disputing that, but I'm not sure we really want it in this case.

Simon


--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

On 02/07/07 at 21:26 -0700, Steve Langasek wrote:
> Lucas has agreed to doing a full archive rebuild with a modified dpkg-dev,
> for comparison with the previous test. A dpkg-dev binary including this
> change can be found at
> .

Hi,

Here are some results:
7299 packages from unstable/main were rebuilt (that's all packages
building non-arch:all packages).
1823 packages were built using 'debian/rules build-arch'.

Of those 1823 packages, 31 packages failed to build. Logs can be found
on http://people.debian.org/~lucas/logs/2007/07/04/ .

Regarding build time, it's difficult to compare, because the most recent
data I have was generated on a different set of cluster nodes, and the
nodes I used for this have a slower network connection. Also, the
mirror's disk is a bottleneck since I was using 55 nodes. But some
packages seem to benefit from using build-arch despite that. See
http://people.debian.org/~lucas/logs.../00impr_bt.txt . Previous
and current build times are the 8th and 9th columns.

There might be others, that /built/ faster, but that took a longer time
to fetch build-deps because of the network/mirror.

The full listing of the results is on
http://people.debian.org/~lucas/logs.../00summary.txt , with the
columns being:
1: package
2, 3, 4: previous, current version, and whether they differ
5, 6, 7: previous, current result, and whether they differ
8, 9, 10: previous, current result, and whether they differ (incl.
margin of error)
11, 12, 13: previous, current reason for build failure, and whether they
differ.

So, to conclude: this change seems like a good idea, with only about 30
packages to fix (but they should probably be fixed anyway). Not so many
packages benefit from it currently, but there was no reason until now for packages
to implement that.
--
| Lucas Nussbaum
| lucas@lucas-nussbaum.net http://www.lucas-nussbaum.net/ |
| jabber: lucas@nussbaum.fr GPG: 1024D/023B3F4F |


--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Hi,

Wouter Verhelst wrote:

> $(shell ls temp-target-* && rm temp-target-*):

> Yes, that's broken, but there are your side effects, and you'll have to
> run this code if you want to make your --has-target work.

Yes, that is exactly what I'm proposing. The same thing happens for -q
mode now.

Simon


--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Wouter Verhelst writes:

> On Mon, Jul 02, 2007 at 03:28:05PM +0200, Simon Richter wrote:
>>> Running debian-rules can always have side effects and can actively
>>> rely on them so a "--has-target" can not be implemented cleanly in
>>> make.
>>
>> I am proposing hooking into the logic that ultimately decides that
>> there is no such target in the Makefile and goes on to print "Don't
>> know how to make 'foo'. Stop.".
>
> $(shell ls temp-target-* && rm temp-target-*):
>
> Yes, that's broken, but there are your side effects, and you'll have to
> run this code if you want to make your --has-target work.

Or just a simple

include debian/rules.gen

while that is generated as needed.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Russ Allbery writes ("Re: Can we require build-arch/indep targets for lenny?"):
> Lo?c Minier writes:
> > Why not promote these to requirements in a particular policy version
> > instead? I fear we will have to list 10 Build-Options in all packages
> > in a couple of years.

This is a much better idea.

> Currently, policy says that it's recommended (the weakest policy
> directive) to support noopt and nostrip. My main concern with increasing
> the strength of that directive is that, depending on how demented the
> upstream build system is, it can be difficult to support these options,
> and since neither is used for regular builds in Debian, they're not
> usually tested and aren't necessary for properly functioning packages.

Surely we are planning to support these options in all packages
eventually ? In a package where it is difficult to separate out the
work for binary-indep, it would be acceptable to say:
binary-indep: binary
binary-arch: binary
binary: build
some stuff
?

I'm tempted to suggest _just_ going by the package's Standards-Version.

Ian.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Ian Jackson writes:
> Russ Allbery writes ("Re: Can we require build-arch/indep targets for lenny?"):

>> Currently, policy says that it's recommended (the weakest policy
>> directive) to support noopt and nostrip. My main concern with
>> increasing the strength of that directive is that, depending on how
>> demented the upstream build system is, it can be difficult to support
>> these options, and since neither is used for regular builds in Debian,
>> they're not usually tested and aren't necessary for properly
>> functioning packages.

> Surely we are planning to support these options in all packages
> eventually ?

It is certainly not clear to me that we're planning on supporting nostrip
and noopt for all packages eventually.

> I'm tempted to suggest _just_ going by the package's Standards-Version.

Based on the arguments I've seen so far, I'm opposed to using the
package's Standards-Version for this purpose. I think it conflates
different meanings of that field and will get us into serious trouble when
it comes to the distinctions between must, should, and recommended.

--
Russ Allbery (rra@debian.org)


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Russ Allbery writes:

>> I'm tempted to suggest _just_ going by the package's Standards-Version.
>
> Based on the arguments I've seen so far, I'm opposed to using the
> package's Standards-Version for this purpose. I think it conflates
> different meanings of that field and will get us into serious trouble when
> it comes to the distinctions between must, should, and recommended.

| Policy 5.6.11 Standards-Version
|
| The most recent version of the standards (the policy manual and
| associated texts) with which the package complies.

This field has exactly this meaning. It says the package followes a
certain version of policy, e.g. the one where now there is a MUST
instead of the previous RECOMMENDS.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Wouter Verhelst writes:

> On Fri, Jun 29, 2007 at 11:11:04PM +0200, Bill Allombert wrote:
>> One of the issue is that tools like sbuild and pbuilder which want to
>> take advantage of the Build-Depends-Indep split needs to know whether
>> dpkg-buildpackage will call debian/rules build or build-arch.
>
> It needs to know no such thing. It just needs to know that if it runs
> "dpkg-buildpackage -b", only .debs will be generated, and if it runs
> "dpkg-buildpackage -B", all debs apart from the _all.debs will be
> generated. How exactly this happens is of no concern to sbuild.

It does when "Build-Depends" is used as specified in policy and not
like it is (brokenly) used now:

Policy 7.6:

| Build-Depends, Build-Conflicts
|
| The Build-Depends and Build-Conflicts fields must be satisfied
| when any of the following targets is invoked: build, clean,
| binary, binary-arch, build-arch, build-indep and binary-indep.
|
| Build-Depends-Indep, Build-Conflicts-Indep
|
| The Build-Depends-Indep and Build-Conflicts-Indep fields must be
| satisfied when any of the following targets is invoked: build,
| build-indep, binary and binary-indep.

That means that is -b is used or if the package does NOT support
build-arch then sbuild must install Build-Depends-Indep as well.

If build-arch remains optional and policy for Build-Depends remains
unchanged then sbuild must now. Making build-arch required or changing
the meaning of Build-Depends to the broken use we have now resolves
this. I favor making build-arch required.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Goswin von Brederlow writes:
> Russ Allbery writes:

>> Based on the arguments I've seen so far, I'm opposed to using the
>> package's Standards-Version for this purpose. I think it conflates
>> different meanings of that field and will get us into serious trouble
>> when it comes to the distinctions between must, should, and
>> recommended.

> | Policy 5.6.11 Standards-Version
> |
> | The most recent version of the standards (the policy manual and
> | associated texts) with which the package complies.

> This field has exactly this meaning. It says the package followes a
> certain version of policy, e.g. the one where now there is a MUST
> instead of the previous RECOMMENDS.

You seem to be ignoring the end of second sentence of my paragraph above,
which I wrote precisely because I anticipated this argument. Could you
respond to it as well? Not every feature we care about is going to be a
must.

I would much prefer to see a new control field that explicitly lists the
supported features. We're going to need that *anyway* for any feature
that's only a should or recommended and not a must (such as supporting
noopt or nostrip), and making every should into a must just so that we can
use this interpretation of Standards-Version is not a solution.

--
Russ Allbery (rra@debian.org)


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Russ Allbery writes:

> Goswin von Brederlow writes:
>> Russ Allbery writes:
>
>>> Based on the arguments I've seen so far, I'm opposed to using the
>>> package's Standards-Version for this purpose. I think it conflates
>>> different meanings of that field and will get us into serious trouble
>>> when it comes to the distinctions between must, should, and
>>> recommended.
>
>> | Policy 5.6.11 Standards-Version
>> |
>> | The most recent version of the standards (the policy manual and
>> | associated texts) with which the package complies.
>
>> This field has exactly this meaning. It says the package followes a
>> certain version of policy, e.g. the one where now there is a MUST
>> instead of the previous RECOMMENDS.
>
> You seem to be ignoring the end of second sentence of my paragraph above,
> which I wrote precisely because I anticipated this argument. Could you
> respond to it as well? Not every feature we care about is going to be a
> must.

I thought you ment that with << ver something is recommended but >>ver
is is must would be a problem.

> I would much prefer to see a new control field that explicitly lists the
> supported features. We're going to need that *anyway* for any feature
> that's only a should or recommended and not a must (such as supporting
> noopt or nostrip), and making every should into a must just so that we can
> use this interpretation of Standards-Version is not a solution.

So far I have not seen anything that would require it.

The build-arch target should be a must so no extra build option flag
needed.

As for the noopt/nostrip feature. What if the source does not support
them? What can you do? Not set them? That is exatly the same as
setting them and having the source not honor them.
Having a build options flag for noopt and nostrip would be purely
informational. It is not like some functionaly gets lost wthout it
unlike the build-arch target.


By all means fight for buiold-options but I still don't see why we
need this for buila-arch/indep targets. There is no good reason to
keep the optional.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Goswin von Brederlow writes:
> Russ Allbery writes:

>> I would much prefer to see a new control field that explicitly lists
>> the supported features. We're going to need that *anyway* for any
>> feature that's only a should or recommended and not a must (such as
>> supporting noopt or nostrip), and making every should into a must just
>> so that we can use this interpretation of Standards-Version is not a
>> solution.

> So far I have not seen anything that would require it.

I think it would be useful to advertise the optional capabilities of a
package (noopt, nostrip, parallel) without forcing people to do trial and
error. I suppose that's not a "require," but it certainly would be nice.

> The build-arch target should be a must so no extra build option flag
> needed.

I really don't think that declaring the majority of packages in Debian
buggy in this fashion is viable, particularly when nearly all packages in
Debian will not benefit from this. My guess is that something on the
order of 1% of packages have a meaningful distinction between build-arch
and build-indep, if that, but that includes some packages that benefit a
*lot*. Wouldn't it be better to only have to work on modifying the
packages that will specifically benefit instead of making every other
package maintainer in Debian add a new target that really isn't meaningful
for their package?

--
Russ Allbery (rra@debian.org)


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Russ Allbery writes:

> Goswin von Brederlow writes:
>> Russ Allbery writes:
>
>>> I would much prefer to see a new control field that explicitly lists
>>> the supported features. We're going to need that *anyway* for any
>>> feature that's only a should or recommended and not a must (such as
>>> supporting noopt or nostrip), and making every should into a must just
>>> so that we can use this interpretation of Standards-Version is not a
>>> solution.
>
>> So far I have not seen anything that would require it.
>
> I think it would be useful to advertise the optional capabilities of a
> package (noopt, nostrip, parallel) without forcing people to do trial and
> error. I suppose that's not a "require," but it certainly would be nice.

As for parallel I don't think build-options is enough. The amount of
parallelism usefull depends too much on the system and package. For
example a simple C source can build fine with -j4 and 256MB ram. But
any c++ source with templates will just swap to death with the same.

In a discussion last year I suggested providing a tool to ask the
system for the prefered parallelism to be used during compile. The
tool would get a few parameters such as what language is used or how
much ram the compiler roughly needs. It would check that against the
systems config and resources and then reply with a parallelism level
the source could use.

>> The build-arch target should be a must so no extra build option flag
>> needed.
>
> I really don't think that declaring the majority of packages in Debian
> buggy in this fashion is viable, particularly when nearly all packages in
> Debian will not benefit from this. My guess is that something on the
> order of 1% of packages have a meaningful distinction between build-arch
> and build-indep, if that, but that includes some packages that benefit a
> *lot*. Wouldn't it be better to only have to work on modifying the
> packages that will specifically benefit instead of making every other
> package maintainer in Debian add a new target that really isn't meaningful
> for their package?

Already 25% of all packages do have the targets and I assume a lot
more than 1% to benefit. If one single Build-Depends can be moved into
Build-Depends-Indep that is already a benefit.

Weigh that against the cost, adding a % to the build target or adding
'build-%: build', for the packages without meaningful distinction. I
just feel that the cost is so small that any smarter solution than
just requiring build-arch/indep tragets is more waste.

And yes, 75% of the archive would become theoretically buggy the
instance we declare it a must. But nothing breaks and nothing is
actually buggy unless the standards-version gets increased by the
maintainer. At least that is how I see it.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Goswin von Brederlow writes:
> Russ Allbery writes:

>> I really don't think that declaring the majority of packages in Debian
>> buggy in this fashion is viable, particularly when nearly all packages
>> in Debian will not benefit from this. My guess is that something on
>> the order of 1% of packages have a meaningful distinction between
>> build-arch and build-indep, if that, but that includes some packages
>> that benefit a *lot*. Wouldn't it be better to only have to work on
>> modifying the packages that will specifically benefit instead of making
>> every other package maintainer in Debian add a new target that really
>> isn't meaningful for their package?

> Already 25% of all packages do have the targets and I assume a lot
> more than 1% to benefit.

I'd be curious to see your reasoning there.

All of my packages have build-arch and build-indep targets. None of them
benefit from them at all. I expect many other people have similarly added
the targets just because, or have the targets provided by a build system
such as CDBS, even though they don't benefit.

> Weigh that against the cost, adding a % to the build target or adding
> 'build-%: build', for the packages without meaningful distinction.

As many people have previously pointed out, that's not the real cost.

I really don't see any justification for forcing all packages to change
when we have a proposed solution that lets only the packages that benefit
change.

--
Russ Allbery (rra@debian.org)


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Russ Allbery writes:

> Goswin von Brederlow writes:
>> Russ Allbery writes:
>
>>> I really don't think that declaring the majority of packages in Debian
>>> buggy in this fashion is viable, particularly when nearly all packages
>>> in Debian will not benefit from this. My guess is that something on
>>> the order of 1% of packages have a meaningful distinction between
>>> build-arch and build-indep, if that, but that includes some packages
>>> that benefit a *lot*. Wouldn't it be better to only have to work on
>>> modifying the packages that will specifically benefit instead of making
>>> every other package maintainer in Debian add a new target that really
>>> isn't meaningful for their package?
>
>> Already 25% of all packages do have the targets and I assume a lot
>> more than 1% to benefit.
>
> I'd be curious to see your reasoning there.
>
> All of my packages have build-arch and build-indep targets. None of them
> benefit from them at all. I expect many other people have similarly added
> the targets just because, or have the targets provided by a build system
> such as CDBS, even though they don't benefit.

For example how many sources have a tex file that they run through
latex for a -doc package?

grep-dctrl -F Build-Depends tetex ftp.de.debian.org_debian_dists_sid_main_source_Sou rces -s Package | wc -l
112

That alone is already 1%.

There might be more involved than just adding the build-arch target to
actualy benefit from it but I see a lot more potential than 1%.

>> Weigh that against the cost, adding a % to the build target or adding
>> 'build-%: build', for the packages without meaningful distinction.
>
> As many people have previously pointed out, that's not the real cost.

There is nothing else costing anything because there is nothing else
to do when the package has no meaningful distinction. You might
disagree what the cost of it is but that is the only thing causing the
cost.

> I really don't see any justification for forcing all packages to change
> when we have a proposed solution that lets only the packages that benefit
> change.

And I don't see the need to invent some field when it is not needed.

Anyway, as long as it gets solved I'm happy. But please people, solve
it.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org