Status for 2.2.0 release

I want to 'code freeze' today the repositories on orocos-toolchain. This means
that each maintainer should merge the last patches and then update the version
number of the package.

After that, it's only documentation updates, or important fixes (announced on
the ML). This provides a small time window for testing before the release of
next Wednesday. The toolchain-2.2 branch will be created (and tagged) when the
sources are released.

Things I'd like to have tested/worked out before release:
- Merge the default 'Makefile' files in ocl and rtt which rosbuild requires.
- make sure that log4cpp is checked out by bootstrap and used by OCL's logger
component. I want the latter to be built by default.
- Check that lua documentation is easily accessible & online. (Urgent!)
- Create an RTT 2.2 cheat-sheet
- Have the exercises checked + add a BSD license there.

There's also a 'transports' repository on gitorious that contains Charles'
Yarp transport code. In the near future, I'd like to move the 'corba', 'ros'
and 'mqueue' transports to there too. I won't release 'transports' yet with
2.2, but It should be ready for 2.3.

Despite all the commotion, it seems 2.2 will be 'ROS-ready', meaning that it
can work in a ROS_PACKAGE_PATH environment (building without installing).
We're still working on the ROS transport & services integration, which should
further mature after the release.

Peter

Ruben Smits's picture

Status for 2.2.0 release

On Monday 06 December 2010 11:56:20 Peter Soetens wrote:
> I want to 'code freeze' today the repositories on orocos-toolchain. This
> means that each maintainer should merge the last patches and then update
> the version number of the package.
>
> After that, it's only documentation updates, or important fixes (announced
> on the ML). This provides a small time window for testing before the
> release of next Wednesday. The toolchain-2.2 branch will be created (and
> tagged) when the sources are released.
>
> Things I'd like to have tested/worked out before release:
> - Merge the default 'Makefile' files in ocl and rtt which rosbuild
> requires.

Is this merged already?

Ruben

> - make sure that log4cpp is checked out by bootstrap and used by
> OCL's logger component. I want the latter to be built by default.
> - Check that lua documentation is easily accessible & online. (Urgent!)
> - Create an RTT 2.2 cheat-sheet
> - Have the exercises checked + add a BSD license there.
>
> There's also a 'transports' repository on gitorious that contains Charles'
> Yarp transport code. In the near future, I'd like to move the 'corba',
> 'ros' and 'mqueue' transports to there too. I won't release 'transports'
> yet with 2.2, but It should be ready for 2.3.
>
> Despite all the commotion, it seems 2.2 will be 'ROS-ready', meaning that
> it can work in a ROS_PACKAGE_PATH environment (building without
> installing). We're still working on the ROS transport & services
> integration, which should further mature after the release.
>
> Peter

Status for 2.2.0 release

On 12/06/2010 11:56 AM, Peter Soetens wrote:
> I want to 'code freeze' today the repositories on orocos-toolchain. This means
> that each maintainer should merge the last patches and then update the version
> number of the package.
What does that mean in practice ?

> There's also a 'transports' repository on gitorious that contains Charles'
> Yarp transport code. In the near future, I'd like to move the 'corba', 'ros'
> and 'mqueue' transports to there too. I won't release 'transports' yet with
> 2.2, but It should be ready for 2.3.
I would rather not build yet another "dump all" package (the "other"
being OCL). Could we move to a "one package per transport" policy ?

Status for 2.2.0 release

On Monday 06 December 2010 12:16:29 Sylvain Joyeux wrote:
> On 12/06/2010 11:56 AM, Peter Soetens wrote:
> > I want to 'code freeze' today the repositories on orocos-toolchain. This
> > means that each maintainer should merge the last patches and then update
> > the version number of the package.
>
> What does that mean in practice ?

I'm updating rtt and ocl to '2.2.0'. The other libraries in the toolchain
should increase their version number too to express 'major feature update',
which you should do. I'll update log4cpp too to version '1.1'.

>
> > There's also a 'transports' repository on gitorious that contains
> > Charles' Yarp transport code. In the near future, I'd like to move the
> > 'corba', 'ros' and 'mqueue' transports to there too. I won't release
> > 'transports' yet with 2.2, but It should be ready for 2.3.
>
> I would rather not build yet another "dump all" package (the "other"
> being OCL). Could we move to a "one package per transport" policy ?

With 'package' you mean 'git repository' ?

I don't see transports as a 'dump all' thing. You're probably worried that the
corba stuff might break when other transports aren't playing nice in the same
repository. But can't we easily solve this using cmake logic which only
builds a transport if the dependencies are present ?

Peter

Status for 2.2.0 release

On 12/06/2010 12:29 PM, Peter Soetens wrote:
> On Monday 06 December 2010 12:16:29 Sylvain Joyeux wrote:
>> On 12/06/2010 11:56 AM, Peter Soetens wrote:
>>> I want to 'code freeze' today the repositories on orocos-toolchain. This
>>> means that each maintainer should merge the last patches and then update
>>> the version number of the package.
>>
>> What does that mean in practice ?
>
> I'm updating rtt and ocl to '2.2.0'. The other libraries in the toolchain
> should increase their version number too to express 'major feature update',
> which you should do. I'll update log4cpp too to version '1.1'.
>
>>
>>> There's also a 'transports' repository on gitorious that contains
>>> Charles' Yarp transport code. In the near future, I'd like to move the
>>> 'corba', 'ros' and 'mqueue' transports to there too. I won't release
>>> 'transports' yet with 2.2, but It should be ready for 2.3.
>>
>> I would rather not build yet another "dump all" package (the "other"
>> being OCL). Could we move to a "one package per transport" policy ?
>
> With 'package' you mean 'git repository' ?
>
> I don't see transports as a 'dump all' thing.
The likelihood that, in one single project, someone would want to use
all transports *at the same time* is very very low. Hence "dump all".

My policy is: things that are used together should belong together.
Otherwise, split packages.

> You're probably worried that the
> corba stuff might break when other transports aren't playing nice in the same
> repository. But can't we easily solve this using cmake logic which only
> builds a transport if the dependencies are present ?
And why should I "behind the scenes" get the LUA scripting just because
lua-dev is installed ? mqueue transport ? Yarp transport ?

In the end, you get a lot of functionality that will not be used at all
at the same time.

Moreover, in both autoproj and rosbuild (I believe for this last one),
functionality is selected by which packages are being built. By putting
a lot of stuff in one package, you remove this ability. I.e. you require
the user to start tuning the CMake options.

Yes, it can be done. Just mentioning that I find the OCL situation
horrible -- which makes me avoid it completely as much as I can -- and
I'd like to avoid having the same situation with the transports package.

Ruben Smits's picture

Status for 2.2.0 release

On Tuesday 07 December 2010 11:07:30 Sylvain Joyeux wrote:
> On 12/07/2010 10:20 AM, Peter Soetens wrote:
> > On Tuesday 07 December 2010 09:55:06 Sylvain Joyeux wrote:
> >> On 12/06/2010 05:02 PM, Peter Soetens wrote:
> >>>>> corba stuff might break when other transports aren't playing nice in
> >>>>> the same repository. But can't we easily solve this using cmake
> >>>>> logic which only builds a transport if the dependencies are present
> >>>>> ?
> >>>>
> >>>> And why should I "behind the scenes" get the LUA scripting just
> >>>> because lua-dev is installed ? mqueue transport ? Yarp transport ?
> >>>
> >>> This is/was actually quite common in the 'configure' days, where
> >>> support for an installed library was automatically added, not
> >>> requireing the user to specify tens of --enable-foo options.
> >>
> >> Yes. And that was already a bad behaviour in my opinion. And I already
> >> split LAAS packages when I wrote autobuild (autoproj's "guts") when the
> >> pieces were not belonging together.
> >>
> >>>> In the end, you get a lot of functionality that will not be used at
> >>>> all at the same time.
> >>>>
> >>>> Moreover, in both autoproj and rosbuild (I believe for this last one),
> >>>> functionality is selected by which packages are being built. By
> >>>> putting a lot of stuff in one package, you remove this ability. I.e.
> >>>> you require the user to start tuning the CMake options.
> >>>
> >>> rosbuild is a cmake macro wrapper. You mean 'rospack'. rospack can
> >>> handle this flawlessly, since one repository my contain multiple
> >>> packages.
> >>
> >> True, forgot that in ros-XXX one package == one manifest.xml. For my
> >> information, can you tell ros-whatever to build one package and not the
> >> other (since they are *all* checked out) ? How ?
> >
> > Since ROS was pre-built on my system, it contained the ROS_NOBUILD tag
> > files, which made it skip 'common dependencies' immediately.
>
> That was not my question.
>
> How do you (can you ?) make sure that
>
> rosmake
>
> without any arguments, does *not* build some packages that are checked out
> ?

If you are inside a stack (defined by having a stack.xml) it will build all
packages inside that stack

If you are inside a package (defined by having a manifest.xml) it will build
that package + all dependencies

If you do rosmake in a directory which is not a package or a stack it will
fail:

rosmake
[ rosmake ] No package or stack specified. And current directory 'tmp' is not
a package name or stack name.
[ rosmake ] Packages requested are: []
[ rosmake ] Logging to
directory/home/rsmits/.ros/rosmake/rosmake_output-20101207-112244
[ rosmake ] Expanded args [] to:
[]
[ rosmake ] ERROR: No arguments could be parsed into valid package or stack
names.

> >> In any case, while autoproj kind-of support this, it is mostly expected
> >> that one package == one repo
> >
> > So this limitation is an issue.
>
> Well, as always, it is a matter of point of view.
>
> I personally don't like your way. Now, you tell me "fix autoproj because
> I want orocos-toolchain to be my way". Well ...

I bumped into robotpkg the other day, how is autoproj different from robotpkg?

> >> Another thing: what is the actual point of keeping them in one
> >> repository if they are, anyway, different cmake packages ...
> >
> > In ROS-speak, they are called 'stacks'. A stack is a logical group of
> > packages, that are always released together. These packages can be
> > perfectly independent or rely on each other, that's the stack's
> > decision. For example, Ruben released our orocos-toolchain git
> > repositories as a single ROS stack. Sometimes a package is only a few
> > files (like xml or scripts). Splitting each package in a git repository
> > would be impractical. So I certainly want to be able to provide multiple
> > packages in one git repository.
>
> That's exactly my point. The transports are *not* a stack !!!! They
> don't get used together (mostly) and therefore will seldom be used in
> the same project.
>
> rtt + ros + ROS-friendly rtt components is a stack
>
> rtt + mqueue + corba + rock is a stack
>
> rtt + yarp + stuff developed using yarp is a stack

I think you are mixing up stacks with installs.

If you look at rosinstall (http://www.ros.org/wiki/rosinstall) it does exactly
what you want, and at the granularity of picking just the packages you want.

> > I believe there are almost too many git repositories right now. For
> > tagging, branching and general 'on-branch-x.y' development, this gets
> > slow, since I have to make sure that each repository is on the right
> > branch.
>
> Well, as always, there's nothing a good tool can't fix. I already
> started on that for rock.
>
> Here's the point: transports and ocl are the tip of the iceberg. The
> rock package library is designed so that people can cherry-pick what
> they want. There are 30+ packages. Think about branching and tagging that.

Again, it feels like you are reinventing rosinstall.

Ruben

Status for 2.2.0 release

On 12/07/2010 11:26 AM, Ruben Smits wrote:
> > Here's the point: transports and ocl are the tip of the iceberg. The
>
> > rock package library is designed so that people can cherry-pick what
>
> > they want. There are 30+ packages. Think about branching and tagging
> that.
>
> Again, it feels like you are reinventing rosinstall.
autoproj is an integrated "ros*". You might or might not like it, but
that's what it is. From the very beginning, *autobuild* checked out code
from multiple repositories, updated them, checked out new branches,
allowed you to compare the state of your checkout with the remote
repositories and so on. With multiple VCS. And multiple package types.
*autobuild* exists since august 2005. Hardly "reinvented".

*autobuild* already did dependency tracking and update/builds partial
parts of your build tree.

*autobuild* did not do stacks, *autoproj* did "stacks" (called package
sets in autoproj speak)

What it did *not* do is get away from the accepted standard of
installing software before you use it. And no rospack in cmake code.
Because I did not like having the build system creep up in cmake/C++
code. And the ROS-specific changes that went into oroGen templates
recently "just to make them work in ros" makes me think that I was
right. Reinventing dependency tracking and making the packages
ROS-specific in the process is plain stupid. Even typekit's C++ code
starts having ROS-specific bits. This makes me shudder.

Moreover, to "reinvent" something, you need to have it in the first
place. autobuild exists since mid-2005 (as far as VCS logs are
concerned). autoproj predates the time I heard about ROS.

About the automatic branching: the tool I am talking about was 10
minutes of coding a script. Hardly a matter of reinventing something.

Ruben Smits's picture

Status for 2.2.0 release

On Tuesday 07 December 2010 12:03:52 Sylvain Joyeux wrote:
> On 12/07/2010 11:26 AM, Ruben Smits wrote:
> > > Here's the point: transports and ocl are the tip of the iceberg. The
> > >
> > > rock package library is designed so that people can cherry-pick what
> > >
> > > they want. There are 30+ packages. Think about branching and tagging
> >
> > that.
> >
> > Again, it feels like you are reinventing rosinstall.
>
> autoproj is an integrated "ros*". You might or might not like it, but
> that's what it is. From the very beginning, *autobuild* checked out code
> from multiple repositories, updated them, checked out new branches,
> allowed you to compare the state of your checkout with the remote
> repositories and so on. With multiple VCS. And multiple package types.
> *autobuild* exists since august 2005. Hardly "reinvented".
>
> *autobuild* already did dependency tracking and update/builds partial
> parts of your build tree.
>
> *autobuild* did not do stacks, *autoproj* did "stacks" (called package
> sets in autoproj speak)

Just to let you know, the following links are broken:

http://rock-robotics.org/api/autoproj/index.html
http://rock-robotics.org/api/autobuild/index.html

-R.

> What it did *not* do is get away from the accepted standard of
> installing software before you use it. And no rospack in cmake code.
> Because I did not like having the build system creep up in cmake/C++
> code. And the ROS-specific changes that went into oroGen templates
> recently "just to make them work in ros" makes me think that I was
> right. Reinventing dependency tracking and making the packages
> ROS-specific in the process is plain stupid. Even typekit's C++ code
> starts having ROS-specific bits. This makes me shudder.
>
> Moreover, to "reinvent" something, you need to have it in the first
> place. autobuild exists since mid-2005 (as far as VCS logs are
> concerned). autoproj predates the time I heard about ROS.
>
> About the automatic branching: the tool I am talking about was 10
> minutes of coding a script. Hardly a matter of reinventing something.

Ruben Smits's picture

Status for 2.2.0 release

On Tuesday 07 December 2010 12:03:52 Sylvain Joyeux wrote:
> On 12/07/2010 11:26 AM, Ruben Smits wrote:
> > > Here's the point: transports and ocl are the tip of the iceberg. The
> > >
> > > rock package library is designed so that people can cherry-pick what
> > >
> > > they want. There are 30+ packages. Think about branching and tagging
> >
> > that.
> >
> > Again, it feels like you are reinventing rosinstall.
>
> autoproj is an integrated "ros*".

Just out of curiosity, what does the * stand for? make,pack,build,install or
all of them at the same time?

> You might or might not like it, but
> that's what it is. From the very beginning, *autobuild* checked out code
> from multiple repositories, updated them, checked out new branches,
> allowed you to compare the state of your checkout with the remote
> repositories and so on. With multiple VCS. And multiple package types.
> *autobuild* exists since august 2005. Hardly "reinvented".

I did not know autobuild was used behind the scene, then again I did not look
to much into autoproj's details. Hmm, maybe we should point the ros people to
autobuild and tell them they are reinventing stuff ;)

> *autobuild* already did dependency tracking and update/builds partial
> parts of your build tree.

So autobuild is more or less like rosmake "package" combined with rospack?

> *autobuild* did not do stacks, *autoproj* did "stacks" (called package
> sets in autoproj speak)

And autoproj is a combination of rosmake and rosinstall?

> What it did *not* do is get away from the accepted standard of
> installing software before you use it. And no rospack in cmake code.
> Because I did not like having the build system creep up in cmake/C++
> code.

I think you are right here, but for me the rosbuild cmake macros just made my
life al lot easier, instantly ... without a glitch, I can not say the same for
autoproj. And my resulting CMakeLists.txt files are really minimal. I do not
have to care about package dependencies myself in my CMakeLists, rosbuild will
make sure all the right flags are imported, without me forgetting one.

> Even typekit's C++ code
> starts having ROS-specific bits. This makes me shudder.

Hmm, can you point at an example of this??

The fact that I can just do import("packagename") in the deployer, and all my
the typekits that I depend on (wherever the are) are automatically loaded (and
only the ones I depend on) is fantastic. This made deployement of applications
dead-easy, from scripting and from XML.

> Moreover, to "reinvent" something, you need to have it in the first
> place. autobuild exists since mid-2005 (as far as VCS logs are
> concerned). autoproj predates the time I heard about ROS.

You're absolutely right here, but I heard about autobuild/proj after I heard
about rosbuild, so it is all about perception.

> About the automatic branching: the tool I am talking about was 10
> minutes of coding a script. Hardly a matter of reinventing something.

Just for the record, just because it is written in 10 minutes does not make it
not reinvented ;)

Ruben

Status for 2.2.0 release

On 12/07/2010 12:30 PM, Ruben Smits wrote:
> On Tuesday 07 December 2010 12:03:52 Sylvain Joyeux wrote:
>> On 12/07/2010 11:26 AM, Ruben Smits wrote:
>>> > Here's the point: transports and ocl are the tip of the iceberg. The
>>> >
>>> > rock package library is designed so that people can cherry-pick what
>>> >
>>> > they want. There are 30+ packages. Think about branching and tagging
>>>
>>> that.
>>>
>>> Again, it feels like you are reinventing rosinstall.
>>
>> autoproj is an integrated "ros*".
>
> Just out of curiosity, what does the * stand for? make,pack,build,install or
> all of them at the same time?
All of them at the same time.

>> *autobuild* already did dependency tracking and update/builds partial
>> parts of your build tree.
>
> So autobuild is more or less like rosmake "package" combined with rospack?
>
>> *autobuild* did not do stacks, *autoproj* did "stacks" (called package
>> sets in autoproj speak)
>
> And autoproj is a combination of rosmake and rosinstall?
Eehhh. In a way.

autobuild is mostly a library that offers services needed to build
something "like" ros* or autoproj. autobuild (the tool) was a DSL-based
system buit on top of it to have something that could be described as
rosmake + rosinstall

autoproj is using autobuild (the library), adding osdeps and "package
sets" (i.e. kind-of stacks) to the mix.

>> What it did *not* do is get away from the accepted standard of
>> installing software before you use it. And no rospack in cmake code.
>> Because I did not like having the build system creep up in cmake/C++
>> code.
>
> I think you are right here, but for me the rosbuild cmake macros just made my
> life al lot easier, instantly ... without a glitch, I can not say the same for
> autoproj. And my resulting CMakeLists.txt files are really minimal. I do not
> have to care about package dependencies myself in my CMakeLists, rosbuild will
> make sure all the right flags are imported, without me forgetting one.
So does pkg-config. Which has been around for ages.

I'm never blaming anyone for *using* ros. I do understand that most
people in software development do not (should not have) to deal with
these issues.

>> Even typekit's C++ code
>> starts having ROS-specific bits. This makes me shudder.
>
> Hmm, can you point at an example of this??
http://gitorious.com/+rock-core-maintainers/orocos-toolchain/rock-orogen...

The typekit code was assuming that the package got installed, which it
is not on ROS.

> The fact that I can just do import("packagename") in the deployer, and all my
> the typekits that I depend on (wherever the are) are automatically loaded (and
> only the ones I depend on) is fantastic. This made deployement of applications
> dead-easy, from scripting and from XML.
And it does work like this on the rock toolchain, using pkg-config.

It is ROS-specific on ROS because ros does install. Otherwise, normal
tools would work fine.

Moreover, it is provided by the RTT, not ROS. ROS has nothing to do with
it (as far as I understood).

>> Moreover, to "reinvent" something, you need to have it in the first
>> place. autobuild exists since mid-2005 (as far as VCS logs are
>> concerned). autoproj predates the time I heard about ROS.
>
> You're absolutely right here, but I heard about autobuild/proj after I heard
> about rosbuild, so it is all about perception.
Agreed
---
Sylvain Joyeux (Dr. Ing.)
Researcher - Space and Security Robotics
DFKI Robotics Innovation Center
Bremen, Robert-Hooke-Straße 5, 28359 Bremen, Germany

Phone: +49 421 218-64136
Fax: +49 421 218-64150
Email: sylvain [dot] joyeux [..] ...

Weitere Informationen: http://www.dfki.de

Ruben Smits's picture

Status for 2.2.0 release

On Tuesday 07 December 2010 14:48:07 Sylvain Joyeux wrote:
> On 12/07/2010 12:30 PM, Ruben Smits wrote:
> > On Tuesday 07 December 2010 12:03:52 Sylvain Joyeux wrote:
> >> On 12/07/2010 11:26 AM, Ruben Smits wrote:
> >>> > Here's the point: transports and ocl are the tip of the iceberg.
> >>> > The
> >>> >
> >>> > rock package library is designed so that people can cherry-pick
> >>> > what
> >>> >
> >>> > they want. There are 30+ packages. Think about branching and
> >>> > tagging
> >>>
> >>> that.
> >>>
> >>> Again, it feels like you are reinventing rosinstall.
> >>
> >> autoproj is an integrated "ros*".
> >
> > Just out of curiosity, what does the * stand for? make,pack,build,install
> > or all of them at the same time?
>
> All of them at the same time.
>
> >> *autobuild* already did dependency tracking and update/builds partial
> >> parts of your build tree.
> >
> > So autobuild is more or less like rosmake "package" combined with
> > rospack?
> >
> >> *autobuild* did not do stacks, *autoproj* did "stacks" (called package
> >> sets in autoproj speak)
> >
> > And autoproj is a combination of rosmake and rosinstall?
>
> Eehhh. In a way.
>
> autobuild is mostly a library that offers services needed to build
> something "like" ros* or autoproj. autobuild (the tool) was a DSL-based
> system buit on top of it to have something that could be described as
> rosmake + rosinstall
>
> autoproj is using autobuild (the library), adding osdeps and "package
> sets" (i.e. kind-of stacks) to the mix.
>
> >> What it did *not* do is get away from the accepted standard of
> >> installing software before you use it. And no rospack in cmake code.
> >> Because I did not like having the build system creep up in cmake/C++
> >> code.
> >
> > I think you are right here, but for me the rosbuild cmake macros just
> > made my life al lot easier, instantly ... without a glitch, I can not
> > say the same for autoproj. And my resulting CMakeLists.txt files are
> > really minimal. I do not have to care about package dependencies myself
> > in my CMakeLists, rosbuild will make sure all the right flags are
> > imported, without me forgetting one.
>
> So does pkg-config. Which has been around for ages.
>
> I'm never blaming anyone for *using* ros. I do understand that most
> people in software development do not (should not have) to deal with
> these issues.
>
> >> Even typekit's C++ code
> >> starts having ROS-specific bits. This makes me shudder.
> >
> > Hmm, can you point at an example of this??
>
> http://gitorious.com/+rock-core-maintainers/orocos-toolchain/rock-orogen/co
> mmit/ab450ade510d17f30c8177e33484e4cdd06d91bb
>
> The typekit code was assuming that the package got installed, which it
> is not on ROS.

How is the fact that stuff does not get installed so "ROS-specific"???

> > The fact that I can just do import("packagename") in the deployer, and
> > all my the typekits that I depend on (wherever the are) are
> > automatically loaded (and only the ones I depend on) is fantastic. This
> > made deployement of applications dead-easy, from scripting and from XML.
>
> And it does work like this on the rock toolchain, using pkg-config.
>
> It is ROS-specific on ROS because ros does install.
Do you mean "does install" or "does not install"?

Because ros by default does not install anything, they don't even have an
install target in there makefiles. And even if stuff does get installed, it
would still work I think.

> Otherwise, normal
> tools would work fine.
>
> Moreover, it is provided by the RTT, not ROS. ROS has nothing to do with
> it (as far as I understood).

AFAIK, the OCL/deployer uses the C++ API of rospack (if available) to find
dependencies at runtime when trying to import a package. If rospack is not
available then you will have to list the imports of all dependent packages
yourself.

Ruben

> >> Moreover, to "reinvent" something, you need to have it in the first
> >> place. autobuild exists since mid-2005 (as far as VCS logs are
> >> concerned). autoproj predates the time I heard about ROS.
> >
> > You're absolutely right here, but I heard about autobuild/proj after I
> > heard about rosbuild, so it is all about perception.
>
> Agreed
> ---
> Sylvain Joyeux (Dr. Ing.)
> Researcher - Space and Security Robotics
> DFKI Robotics Innovation Center
> Bremen, Robert-Hooke-Straße 5, 28359 Bremen, Germany
>
> Phone: +49 421 218-64136
> Fax: +49 421 218-64150
> Email: sylvain [dot] joyeux [..] ...
>
> Weitere Informationen: http://www.dfki.de

Status for 2.2.0 release

On Monday 06 December 2010 13:31:13 Sylvain Joyeux wrote:
> On 12/06/2010 12:29 PM, Peter Soetens wrote:
> > On Monday 06 December 2010 12:16:29 Sylvain Joyeux wrote:
> >> On 12/06/2010 11:56 AM, Peter Soetens wrote:
> >>> I want to 'code freeze' today the repositories on orocos-toolchain.
> >>> This means that each maintainer should merge the last patches and then
> >>> update the version number of the package.
> >>
> >> What does that mean in practice ?
> >
> > I'm updating rtt and ocl to '2.2.0'. The other libraries in the toolchain
> > should increase their version number too to express 'major feature
> > update', which you should do. I'll update log4cpp too to version '1.1'.
> >
> >>> There's also a 'transports' repository on gitorious that contains
> >>> Charles' Yarp transport code. In the near future, I'd like to move the
> >>> 'corba', 'ros' and 'mqueue' transports to there too. I won't release
> >>> 'transports' yet with 2.2, but It should be ready for 2.3.
> >>
> >> I would rather not build yet another "dump all" package (the "other"
> >> being OCL). Could we move to a "one package per transport" policy ?
> >
> > With 'package' you mean 'git repository' ?
> >
> > I don't see transports as a 'dump all' thing.
>
> The likelihood that, in one single project, someone would want to use
> all transports *at the same time* is very very low. Hence "dump all".

That's true.

>
> My policy is: things that are used together should belong together.
> Otherwise, split packages.

If we define 'package' by "contains a 'manifest.xml' file", then there's no harm
in hosting multiple packages in one git repository. We could leave the top-
level transports directory empty, only containing subdirs.

>
> > You're probably worried that the
> > corba stuff might break when other transports aren't playing nice in the
> > same repository. But can't we easily solve this using cmake logic which
> > only builds a transport if the dependencies are present ?
>
> And why should I "behind the scenes" get the LUA scripting just because
> lua-dev is installed ? mqueue transport ? Yarp transport ?

This is/was actually quite common in the 'configure' days, where support for an
installed library was automatically added, not requireing the user to specify
tens of --enable-foo options.

>
> In the end, you get a lot of functionality that will not be used at all
> at the same time.
>
> Moreover, in both autoproj and rosbuild (I believe for this last one),
> functionality is selected by which packages are being built. By putting
> a lot of stuff in one package, you remove this ability. I.e. you require
> the user to start tuning the CMake options.

rosbuild is a cmake macro wrapper. You mean 'rospack'. rospack can handle this
flawlessly, since one repository my contain multiple packages.

>
> Yes, it can be done. Just mentioning that I find the OCL situation
> horrible -- which makes me avoid it completely as much as I can -- and
> I'd like to avoid having the same situation with the transports package.

OCL is indeed one package. We are getting pretty close to 'packetizing' ocl if
we would want to.

Peter

Status for 2.2.0 release

On 12/06/2010 05:02 PM, Peter Soetens wrote:
>>> corba stuff might break when other transports aren't playing nice in the
>>> same repository. But can't we easily solve this using cmake logic which
>>> only builds a transport if the dependencies are present ?
>>
>> And why should I "behind the scenes" get the LUA scripting just because
>> lua-dev is installed ? mqueue transport ? Yarp transport ?
>
> This is/was actually quite common in the 'configure' days, where support for an
> installed library was automatically added, not requireing the user to specify
> tens of --enable-foo options.
Yes. And that was already a bad behaviour in my opinion. And I already
split LAAS packages when I wrote autobuild (autoproj's "guts") when the
pieces were not belonging together.

>> In the end, you get a lot of functionality that will not be used at all
>> at the same time.
>>
>> Moreover, in both autoproj and rosbuild (I believe for this last one),
>> functionality is selected by which packages are being built. By putting
>> a lot of stuff in one package, you remove this ability. I.e. you require
>> the user to start tuning the CMake options.
>
> rosbuild is a cmake macro wrapper. You mean 'rospack'. rospack can handle this
> flawlessly, since one repository my contain multiple packages.
True, forgot that in ros-XXX one package == one manifest.xml. For my
information, can you tell ros-whatever to build one package and not the
other (since they are *all* checked out) ? How ?

In any case, while autoproj kind-of support this, it is mostly expected
that one package == one repo

Another thing: what is the actual point of keeping them in one
repository if they are, anyway, different cmake packages ...

>> Yes, it can be done. Just mentioning that I find the OCL situation
>> horrible -- which makes me avoid it completely as much as I can -- and
>> I'd like to avoid having the same situation with the transports package.
>
> OCL is indeed one package. We are getting pretty close to 'packetizing' ocl if
> we would want to.
There's always a huge gap between "we can do it if we want to" and "it
is done". You know that.

Sylvain

Status for 2.2.0 release

On Tue, Dec 07, 2010 at 09:55:06AM +0100, Sylvain Joyeux wrote:
> On 12/06/2010 05:02 PM, Peter Soetens wrote:
> >>> corba stuff might break when other transports aren't playing nice in the
> >>> same repository. But can't we easily solve this using cmake logic which
> >>> only builds a transport if the dependencies are present ?
> >>
> >> And why should I "behind the scenes" get the LUA scripting just because
> >> lua-dev is installed ? mqueue transport ? Yarp transport ?
> >
> > This is/was actually quite common in the 'configure' days, where support for an
> > installed library was automatically added, not requireing the user to specify
> > tens of --enable-foo options.
> Yes. And that was already a bad behaviour in my opinion. And I already
> split LAAS packages when I wrote autobuild (autoproj's "guts") when the
> pieces were not belonging together.

BTW, why didn't you use robotpkg back then? Just curious.

Markus

Status for 2.2.0 release

On 12/07/2010 12:59 PM, Markus Klotzbuecher wrote:
>> Yes. And that was already a bad behaviour in my opinion. And I already
>> split LAAS packages when I wrote autobuild (autoproj's "guts") when the
>> pieces were not belonging together.
>
> BTW, why didn't you use robotpkg back then? Just curious.

Because robotpkg did not exist. robotpkg started when I left. I don't
know why they did not use autobuild at that time -- as some other people
at LAAS were (and still are) using it.

Sylvain

Status for 2.2.0 release

On Tuesday 07 December 2010 09:55:06 Sylvain Joyeux wrote:
> On 12/06/2010 05:02 PM, Peter Soetens wrote:
> >>> corba stuff might break when other transports aren't playing nice in
> >>> the same repository. But can't we easily solve this using cmake logic
> >>> which only builds a transport if the dependencies are present ?
> >>
> >> And why should I "behind the scenes" get the LUA scripting just because
> >> lua-dev is installed ? mqueue transport ? Yarp transport ?
> >
> > This is/was actually quite common in the 'configure' days, where support
> > for an installed library was automatically added, not requireing the
> > user to specify tens of --enable-foo options.
>
> Yes. And that was already a bad behaviour in my opinion. And I already
> split LAAS packages when I wrote autobuild (autoproj's "guts") when the
> pieces were not belonging together.
>
> >> In the end, you get a lot of functionality that will not be used at all
> >> at the same time.
> >>
> >> Moreover, in both autoproj and rosbuild (I believe for this last one),
> >> functionality is selected by which packages are being built. By putting
> >> a lot of stuff in one package, you remove this ability. I.e. you require
> >> the user to start tuning the CMake options.
> >
> > rosbuild is a cmake macro wrapper. You mean 'rospack'. rospack can handle
> > this flawlessly, since one repository my contain multiple packages.
>
> True, forgot that in ros-XXX one package == one manifest.xml. For my
> information, can you tell ros-whatever to build one package and not the
> other (since they are *all* checked out) ? How ?

rosmake 'foo'
-> will use 'rospack find' to locate a directory 'foo' in the ROS_PACKAGE_PATH
which contains a manifest.xml file.
-> builds all dependencies of 'foo', according to manifest.xml
-> runs make in that directory, uses the one-line 'Makefile' Ruben has posted
before.

For example:
$ rosmake testros
[ rosmake ] Packages requested are: ['testros']
[ rosmake ] Logging to
directory/home/kaltan/.ros/rosmake/rosmake_output-20101207-100702
[ rosmake ] Expanded args ['testros'] to:
['testros']
[ rosmake ] Checking rosdeps compliance for packages testros. This may take a
few seconds.
Failed to find rosdep omniorb for package testros on OS:ubuntu version:10.04
Failed to find rosdep libreadline for package testros on OS:ubuntu
version:10.04
WARNING: Rosdeps [u'omniorb', u'libreadline'] could not be resolved
[ rosmake ] rosdep check passed all system dependencies in packages
[rosmake-0] Starting >>> rtt [ make ]
[rosmake-0] Finished <<< rtt No Makefile in package rtt
[rosmake-1] Starting >>> roslib [ make ]
[rosmake-1] Finished <<< roslib ROS_NOBUILD in package roslib
[rosmake-2] Starting >>> roslang [ make ]
[rosmake-2] Finished <<< roslang ROS_NOBUILD in package roslang
No Makefile in package roslang
[rosmake-3] Starting >>> xmlrpcpp [ make ]
[rosmake-3] Finished <<< xmlrpcpp ROS_NOBUILD in package xmlrpcpp
[rosmake-0] Starting >>> ocl [ make ]
[rosmake-1] Starting >>> rosconsole [ make ]
[rosmake-0] Finished <<< ocl No Makefile in package ocl
[rosmake-1] Finished <<< rosconsole ROS_NOBUILD in package rosconsole
[rosmake-1] Starting >>> std_msgs [ make ]
[rosmake-4] Starting >>> roscpp [ make ]
[rosmake-4] Finished <<< roscpp ROS_NOBUILD in package roscpp
[rosmake-1] Finished <<< std_msgs ROS_NOBUILD in package std_msgs
[rosmake-4] Starting >>> testros [ make ]
[rosmake-3] Starting >>> rosout [ make ]
[rosmake-3] Finished <<< rosout ROS_NOBUILD in package rosout
[rosmake-4] Finished <<< testros [PASS] [ 16.53 seconds ]
[ rosmake ] Results:
[ rosmake ] Built 12 packages with 0 failures.
[ rosmake ] Summary output to directory
[ rosmake ] /home/kaltan/.ros/rosmake/rosmake_output-20101207-100702

Since ROS was pre-built on my system, it contained the ROS_NOBUILD tag files,
which made it skip 'common dependencies' immediately.

>
> In any case, while autoproj kind-of support this, it is mostly expected
> that one package == one repo

So this limitation is an issue.

>
> Another thing: what is the actual point of keeping them in one
> repository if they are, anyway, different cmake packages ...

In ROS-speak, they are called 'stacks'. A stack is a logical group of
packages, that are always released together. These packages can be perfectly
independent or rely on each other, that's the stack's decision. For example,
Ruben released our orocos-toolchain git repositories as a single ROS stack.
Sometimes a package is only a few files (like xml or scripts). Splitting each
package in a git repository would be impractical. So I certainly want to be
able to provide multiple packages in one git repository.

I believe there are almost too many git repositories right now. For tagging,
branching and general 'on-branch-x.y' development, this gets slow, since I
have to make sure that each repository is on the right branch.

>
> >> Yes, it can be done. Just mentioning that I find the OCL situation
> >> horrible -- which makes me avoid it completely as much as I can -- and
> >> I'd like to avoid having the same situation with the transports package.
> >
> > OCL is indeed one package. We are getting pretty close to 'packetizing'
> > ocl if we would want to.
>
> There's always a huge gap between "we can do it if we want to" and "it
> is done". You know that.

Well, I would at least keep them in the same repository. In my view OCL is
pretty thin and focused already.

Peter

Status for 2.2.0 release

On 12/07/2010 10:20 AM, Peter Soetens wrote:
> On Tuesday 07 December 2010 09:55:06 Sylvain Joyeux wrote:
>> On 12/06/2010 05:02 PM, Peter Soetens wrote:
>>>>> corba stuff might break when other transports aren't playing nice in
>>>>> the same repository. But can't we easily solve this using cmake logic
>>>>> which only builds a transport if the dependencies are present ?
>>>>
>>>> And why should I "behind the scenes" get the LUA scripting just because
>>>> lua-dev is installed ? mqueue transport ? Yarp transport ?
>>>
>>> This is/was actually quite common in the 'configure' days, where support
>>> for an installed library was automatically added, not requireing the
>>> user to specify tens of --enable-foo options.
>>
>> Yes. And that was already a bad behaviour in my opinion. And I already
>> split LAAS packages when I wrote autobuild (autoproj's "guts") when the
>> pieces were not belonging together.
>>
>>>> In the end, you get a lot of functionality that will not be used at all
>>>> at the same time.
>>>>
>>>> Moreover, in both autoproj and rosbuild (I believe for this last one),
>>>> functionality is selected by which packages are being built. By putting
>>>> a lot of stuff in one package, you remove this ability. I.e. you require
>>>> the user to start tuning the CMake options.
>>>
>>> rosbuild is a cmake macro wrapper. You mean 'rospack'. rospack can handle
>>> this flawlessly, since one repository my contain multiple packages.
>>
>> True, forgot that in ros-XXX one package == one manifest.xml. For my
>> information, can you tell ros-whatever to build one package and not the
>> other (since they are *all* checked out) ? How ?
>
> Since ROS was pre-built on my system, it contained the ROS_NOBUILD tag files,
> which made it skip 'common dependencies' immediately.
That was not my question.

How do you (can you ?) make sure that

rosmake

without any arguments, does *not* build some packages that are checked out ?

>> In any case, while autoproj kind-of support this, it is mostly expected
>> that one package == one repo
>
> So this limitation is an issue.
Well, as always, it is a matter of point of view.

I personally don't like your way. Now, you tell me "fix autoproj because
I want orocos-toolchain to be my way". Well ...

>> Another thing: what is the actual point of keeping them in one
>> repository if they are, anyway, different cmake packages ...
>
> In ROS-speak, they are called 'stacks'. A stack is a logical group of
> packages, that are always released together. These packages can be perfectly
> independent or rely on each other, that's the stack's decision. For example,
> Ruben released our orocos-toolchain git repositories as a single ROS stack.
> Sometimes a package is only a few files (like xml or scripts). Splitting each
> package in a git repository would be impractical. So I certainly want to be
> able to provide multiple packages in one git repository.
That's exactly my point. The transports are *not* a stack !!!! They
don't get used together (mostly) and therefore will seldom be used in
the same project.

rtt + ros + ROS-friendly rtt components is a stack

rtt + mqueue + corba + rock is a stack

rtt + yarp + stuff developed using yarp is a stack

> I believe there are almost too many git repositories right now. For tagging,
> branching and general 'on-branch-x.y' development, this gets slow, since I
> have to make sure that each repository is on the right branch.
Well, as always, there's nothing a good tool can't fix. I already
started on that for rock.

Here's the point: transports and ocl are the tip of the iceberg. The
rock package library is designed so that people can cherry-pick what
they want. There are 30+ packages. Think about branching and tagging that.

Status for 2.2.0 release

On Tuesday 07 December 2010 11:07:30 Sylvain Joyeux wrote:
> On 12/07/2010 10:20 AM, Peter Soetens wrote:
> > On Tuesday 07 December 2010 09:55:06 Sylvain Joyeux wrote:
> >> On 12/06/2010 05:02 PM, Peter Soetens wrote:
> >>>>> corba stuff might break when other transports aren't playing nice in
> >>>>> the same repository. But can't we easily solve this using cmake
> >>>>> logic which only builds a transport if the dependencies are present
> >>>>> ?
> >>>>
> >>>> And why should I "behind the scenes" get the LUA scripting just
> >>>> because lua-dev is installed ? mqueue transport ? Yarp transport ?
> >>>
> >>> This is/was actually quite common in the 'configure' days, where
> >>> support for an installed library was automatically added, not
> >>> requireing the user to specify tens of --enable-foo options.
> >>
> >> Yes. And that was already a bad behaviour in my opinion. And I already
> >> split LAAS packages when I wrote autobuild (autoproj's "guts") when the
> >> pieces were not belonging together.
> >>
> >>>> In the end, you get a lot of functionality that will not be used at
> >>>> all at the same time.
> >>>>
> >>>> Moreover, in both autoproj and rosbuild (I believe for this last one),
> >>>> functionality is selected by which packages are being built. By
> >>>> putting a lot of stuff in one package, you remove this ability. I.e.
> >>>> you require the user to start tuning the CMake options.
> >>>
> >>> rosbuild is a cmake macro wrapper. You mean 'rospack'. rospack can
> >>> handle this flawlessly, since one repository my contain multiple
> >>> packages.
> >>
> >> True, forgot that in ros-XXX one package == one manifest.xml. For my
> >> information, can you tell ros-whatever to build one package and not the
> >> other (since they are *all* checked out) ? How ?
> >
> > Since ROS was pre-built on my system, it contained the ROS_NOBUILD tag
> > files, which made it skip 'common dependencies' immediately.
>
> That was not my question.
>
> How do you (can you ?) make sure that
>
> rosmake
>
> without any arguments, does *not* build some packages that are checked out
> ?

The work flow demands that you always specify a target. Either implicitly,
because you're in a given directory (as Ruben writes), or explicitly. Some
'packages' only contain a manifest.xml file + a tons of dependencies in order
to define such a target. Packages should be light.

>
> >> In any case, while autoproj kind-of support this, it is mostly expected
> >> that one package == one repo
> >
> > So this limitation is an issue.
>
> Well, as always, it is a matter of point of view.
>
> I personally don't like your way. Now, you tell me "fix autoproj because
> I want orocos-toolchain to be my way". Well ...

The issue is that I don't want to break autoproj. I didn't say that you need
to change it.

>
> >> Another thing: what is the actual point of keeping them in one
> >> repository if they are, anyway, different cmake packages ...
> >
> > In ROS-speak, they are called 'stacks'. A stack is a logical group of
> > packages, that are always released together. These packages can be
> > perfectly independent or rely on each other, that's the stack's
> > decision. For example, Ruben released our orocos-toolchain git
> > repositories as a single ROS stack. Sometimes a package is only a few
> > files (like xml or scripts). Splitting each package in a git repository
> > would be impractical. So I certainly want to be able to provide multiple
> > packages in one git repository.
>
> That's exactly my point. The transports are *not* a stack !!!! They
> don't get used together (mostly) and therefore will seldom be used in
> the same project.
>
> rtt + ros + ROS-friendly rtt components is a stack
>
> rtt + mqueue + corba + rock is a stack
>
> rtt + yarp + stuff developed using yarp is a stack

I can follow this argument too. But as I wrote, what makes up a stack is up to
the maintainer. A stack does not mean that all of the stack is built when you
'rosmake' a certain target. It will only be built when you 'rosmake' the stack
itself (Ruben, correct me if I'm wrong). We are definately talking on different
levels. As I undertand it, your major 'user' concern is that you don't want to
build or depend on stuff that you don't use. I completely agree. The way we're
doing it is not by putting each unit in a git repository (as you seem to do),
but to take an orthogonal approach to that. A package defines a unit of
compilation, and packages depend on each other. So in the 'worst' case, you
have checked out too much source code, but you will only build it if your make
target depends on it.

>
> > I believe there are almost too many git repositories right now. For
> > tagging, branching and general 'on-branch-x.y' development, this gets
> > slow, since I have to make sure that each repository is on the right
> > branch.
>
> Well, as always, there's nothing a good tool can't fix. I already
> started on that for rock.
>
> Here's the point: transports and ocl are the tip of the iceberg. The
> rock package library is designed so that people can cherry-pick what
> they want. There are 30+ packages. Think about branching and tagging that.

30 packages is nothing. Mobile robots are driving around with hundreds of
them. The key is that they don't tie a single package to a single repository,
such that branching and tagging are largely (completely) orthogonal to that.

We don't 'cherry-pick'. We only specify a top-level target, and the dependency
resolution does the rest.

Peter

Status for 2.2.0 release

On 12/07/2010 12:24 PM, Peter Soetens wrote:
>>>> In any case, while autoproj kind-of support this, it is mostly expected
>>>> that one package == one repo
>>>
>>> So this limitation is an issue.
>>
>> Well, as always, it is a matter of point of view.
>>
>> I personally don't like your way. Now, you tell me "fix autoproj because
>> I want orocos-toolchain to be my way". Well ...
>
> The issue is that I don't want to break autoproj. I didn't say that you need
> to change it.
Ah ... Sorry for being snappy. I am a bit on edge these last days
because of a few personal issues. Sorry again.

>>>> Another thing: what is the actual point of keeping them in one
>>>> repository if they are, anyway, different cmake packages ...
>>>
>>> In ROS-speak, they are called 'stacks'. A stack is a logical group of
>>> packages, that are always released together. These packages can be
>>> perfectly independent or rely on each other, that's the stack's
>>> decision. For example, Ruben released our orocos-toolchain git
>>> repositories as a single ROS stack. Sometimes a package is only a few
>>> files (like xml or scripts). Splitting each package in a git repository
>>> would be impractical. So I certainly want to be able to provide multiple
>>> packages in one git repository.
>>
>> That's exactly my point. The transports are *not* a stack !!!! They
>> don't get used together (mostly) and therefore will seldom be used in
>> the same project.
>>
>> rtt + ros + ROS-friendly rtt components is a stack
>>
>> rtt + mqueue + corba + rock is a stack
>>
>> rtt + yarp + stuff developed using yarp is a stack
>
> I can follow this argument too. But as I wrote, what makes up a stack is up to
> the maintainer. A stack does not mean that all of the stack is built when you
> 'rosmake' a certain target. It will only be built when you 'rosmake' the stack
> itself (Ruben, correct me if I'm wrong). We are definately talking on different
> levels.
> As I undertand it, your major 'user' concern is that you don't want to
> build or depend on stuff that you don't use. I completely agree. The way we're
> doing it is not by putting each unit in a git repository (as you seem to do),
> but to take an orthogonal approach to that. A package defines a unit of
> compilation, and packages depend on each other. So in the 'worst' case, you
> have checked out too much source code, but you will only build it if your make
> target depends on it.
Fair enough. I personally don't see the point, but I can at least admit
it is a way ;-)

>> Here's the point: transports and ocl are the tip of the iceberg. The
>> rock package library is designed so that people can cherry-pick what
>> they want. There are 30+ packages. Think about branching and tagging that.
>
> 30 packages is nothing. Mobile robots are driving around with hundreds of
> them. The key is that they don't tie a single package to a single repository,
> such that branching and tagging are largely (completely) orthogonal to that.
We do agree on the scale. But another solution is to make a tool that
does the branching and tagging.

> We don't 'cherry-pick'. We only specify a top-level target, and the dependency
> resolution does the rest.
Two workflows here:

* you have a stack that defines all the targets you need (as a
project). Then you can run rosmake in that stack to get it to build.
In autoproj, it would mean that you have a manifest that
lists all the "toplevel" packages you want -- and then dependency
resolution does the rest
* you don't have such stack. Then, you actually *need* to list
explicitely what you want ... i.e. create one.

Am I wrong ?

Status for 2.2.0 release

On Tuesday 07 December 2010 12:49:01 Sylvain Joyeux wrote:
> On 12/07/2010 12:24 PM, Peter Soetens wrote:
> >>>> In any case, while autoproj kind-of support this, it is mostly
> >>>> expected that one package == one repo
> >>>
> >>> So this limitation is an issue.
> >>
> >> Well, as always, it is a matter of point of view.
> >>
> >> I personally don't like your way. Now, you tell me "fix autoproj because
> >> I want orocos-toolchain to be my way". Well ...
> >
> > The issue is that I don't want to break autoproj. I didn't say that you
> > need to change it.
>
> Ah ... Sorry for being snappy. I am a bit on edge these last days
> because of a few personal issues. Sorry again.
>
> >>>> Another thing: what is the actual point of keeping them in one
> >>>> repository if they are, anyway, different cmake packages ...
> >>>
> >>> In ROS-speak, they are called 'stacks'. A stack is a logical group of
> >>> packages, that are always released together. These packages can be
> >>> perfectly independent or rely on each other, that's the stack's
> >>> decision. For example, Ruben released our orocos-toolchain git
> >>> repositories as a single ROS stack. Sometimes a package is only a few
> >>> files (like xml or scripts). Splitting each package in a git repository
> >>> would be impractical. So I certainly want to be able to provide
> >>> multiple packages in one git repository.
> >>
> >> That's exactly my point. The transports are *not* a stack !!!! They
> >> don't get used together (mostly) and therefore will seldom be used in
> >> the same project.
> >>
> >> rtt + ros + ROS-friendly rtt components is a stack
> >>
> >> rtt + mqueue + corba + rock is a stack
> >>
> >> rtt + yarp + stuff developed using yarp is a stack
> >
> > I can follow this argument too. But as I wrote, what makes up a stack is
> > up to the maintainer. A stack does not mean that all of the stack is
> > built when you 'rosmake' a certain target. It will only be built when
> > you 'rosmake' the stack itself (Ruben, correct me if I'm wrong). We are
> > definately talking on different levels.
> > As I undertand it, your major 'user' concern is that you don't want to
> > build or depend on stuff that you don't use. I completely agree. The way
> > we're doing it is not by putting each unit in a git repository (as you
> > seem to do), but to take an orthogonal approach to that. A package
> > defines a unit of compilation, and packages depend on each other. So in
> > the 'worst' case, you have checked out too much source code, but you
> > will only build it if your make target depends on it.
>
> Fair enough. I personally don't see the point, but I can at least admit
> it is a way ;-)
>
> >> Here's the point: transports and ocl are the tip of the iceberg. The
> >> rock package library is designed so that people can cherry-pick what
> >> they want. There are 30+ packages. Think about branching and tagging
> >> that.
> >
> > 30 packages is nothing. Mobile robots are driving around with hundreds of
> > them. The key is that they don't tie a single package to a single
> > repository, such that branching and tagging are largely (completely)
> > orthogonal to that.
>
> We do agree on the scale. But another solution is to make a tool that
> does the branching and tagging.
>
> > We don't 'cherry-pick'. We only specify a top-level target, and the
> > dependency resolution does the rest.
>
> Two workflows here:
>
> * you have a stack that defines all the targets you need (as a
> project). Then you can run rosmake in that stack to get it to build.
> In autoproj, it would mean that you have a manifest that
> lists all the "toplevel" packages you want -- and then dependency
> resolution does the rest

Yes. I was thinking the same way when searching for similarities.

> * you don't have such stack. Then, you actually *need* to list
> explicitely what you want ... i.e. create one.
>
> Am I wrong ?

No. A ros stack indeed resembles an autoproj layout. The primary reason that
they invented stacks was because stacks are released, not packages. Ie a stack
has a version number, a package hasn't. One Debian package is one stack etc.

For your information, there are lengthy discussions going on on the ros-
developers mailing list about improving the build system such that it won't be
ROS specific anymore but general to many projects. They are writing stuff up on
wiki pages and REPs

For rosbuild (the cmake macros):
http://www.ros.org/wiki/rosbuild/Reviews/Rosbuild%202%20API%20Review

For the build system:
http://www.ros.org/reps/rep-0102.html

For separating the build system out of the rest:
http://www.ros.org/reps/rep-0100.html

Peter

Status for 2.2.0 release

On 12/07/2010 10:01 PM, Peter Soetens wrote:
> On Tuesday 07 December 2010 12:49:01 Sylvain Joyeux wrote:
>> On 12/07/2010 12:24 PM, Peter Soetens wrote:
>>>>>> In any case, while autoproj kind-of support this, it is mostly
>>>>>> expected that one package == one repo
>>>>>
>>>>> So this limitation is an issue.
>>>>
>>>> Well, as always, it is a matter of point of view.
>>>>
>>>> I personally don't like your way. Now, you tell me "fix autoproj because
>>>> I want orocos-toolchain to be my way". Well ...
>>>
>>> The issue is that I don't want to break autoproj. I didn't say that you
>>> need to change it.
>>
>> Ah ... Sorry for being snappy. I am a bit on edge these last days
>> because of a few personal issues. Sorry again.
>>
>>>>>> Another thing: what is the actual point of keeping them in one
>>>>>> repository if they are, anyway, different cmake packages ...
>>>>>
>>>>> In ROS-speak, they are called 'stacks'. A stack is a logical group of
>>>>> packages, that are always released together. These packages can be
>>>>> perfectly independent or rely on each other, that's the stack's
>>>>> decision. For example, Ruben released our orocos-toolchain git
>>>>> repositories as a single ROS stack. Sometimes a package is only a few
>>>>> files (like xml or scripts). Splitting each package in a git repository
>>>>> would be impractical. So I certainly want to be able to provide
>>>>> multiple packages in one git repository.
>>>>
>>>> That's exactly my point. The transports are *not* a stack !!!! They
>>>> don't get used together (mostly) and therefore will seldom be used in
>>>> the same project.
>>>>
>>>> rtt + ros + ROS-friendly rtt components is a stack
>>>>
>>>> rtt + mqueue + corba + rock is a stack
>>>>
>>>> rtt + yarp + stuff developed using yarp is a stack
>>>
>>> I can follow this argument too. But as I wrote, what makes up a stack is
>>> up to the maintainer. A stack does not mean that all of the stack is
>>> built when you 'rosmake' a certain target. It will only be built when
>>> you 'rosmake' the stack itself (Ruben, correct me if I'm wrong). We are
>>> definately talking on different levels.
>>> As I undertand it, your major 'user' concern is that you don't want to
>>> build or depend on stuff that you don't use. I completely agree. The way
>>> we're doing it is not by putting each unit in a git repository (as you
>>> seem to do), but to take an orthogonal approach to that. A package
>>> defines a unit of compilation, and packages depend on each other. So in
>>> the 'worst' case, you have checked out too much source code, but you
>>> will only build it if your make target depends on it.
>>
>> Fair enough. I personally don't see the point, but I can at least admit
>> it is a way ;-)
>>
>>>> Here's the point: transports and ocl are the tip of the iceberg. The
>>>> rock package library is designed so that people can cherry-pick what
>>>> they want. There are 30+ packages. Think about branching and tagging
>>>> that.
>>>
>>> 30 packages is nothing. Mobile robots are driving around with hundreds of
>>> them. The key is that they don't tie a single package to a single
>>> repository, such that branching and tagging are largely (completely)
>>> orthogonal to that.
>>
>> We do agree on the scale. But another solution is to make a tool that
>> does the branching and tagging.
>>
>>> We don't 'cherry-pick'. We only specify a top-level target, and the
>>> dependency resolution does the rest.
>>
>> Two workflows here:
>>
>> * you have a stack that defines all the targets you need (as a
>> project). Then you can run rosmake in that stack to get it to build.
>> In autoproj, it would mean that you have a manifest that
>> lists all the "toplevel" packages you want -- and then dependency
>> resolution does the rest
>
> Yes. I was thinking the same way when searching for similarities.
>
>> * you don't have such stack. Then, you actually *need* to list
>> explicitely what you want ... i.e. create one.
>>
>> Am I wrong ?
>
> No. A ros stack indeed resembles an autoproj layout. The primary reason that
> they invented stacks was because stacks are released, not packages. Ie a stack
> has a version number, a package hasn't. One Debian package is one stack etc.
>
> For your information, there are lengthy discussions going on on the ros-
> developers mailing list about improving the build system such that it won't be
> ROS specific anymore but general to many projects. They are writing stuff up on
> wiki pages and REPs
>
> For rosbuild (the cmake macros):
> http://www.ros.org/wiki/rosbuild/Reviews/Rosbuild%202%20API%20Review
>
> For the build system:
> http://www.ros.org/reps/rep-0102.html
>
> For separating the build system out of the rest:
> http://www.ros.org/reps/rep-0100.html

Good to know.

Now, I'll be honest: it won't change anything on the rock side. We have
a well-working build system and we don't plan to change it anytime soon.

In any case, if they *do* design a build system that can (finally)
accept any kind of decently designed package, then
- they will be able to integrate any rock package out of the box in
ROS stacks (and good for them !)
- I will be able to build any ROSbuild package out of the box into
autoproj (and good for me)

From what I can read, they assume that "if a package is a CMake
package, then it is rosbuild controlled". So I fear that it will not happen.

Well, at least they realized that an install target is an actually
useful thing.

Status for 2.2.0 release

On Mon, 6 Dec 2010, Peter Soetens wrote:

[...]
>> Yes, it can be done. Just mentioning that I find the OCL situation
>> horrible -- which makes me avoid it completely as much as I can -- and
>> I'd like to avoid having the same situation with the transports package.
>
> OCL is indeed one package. We are getting pretty close to 'packetizing' ocl if
> we would want to.

Let's put this on the agenda of the developers as a priority item for the
next release... OCL requires a very severe refactoring anyway...

> Peter

Herman

Status for 2.2.0 release

On Mon, Dec 06, 2010 at 01:31:13PM +0100, Sylvain Joyeux wrote:
> On 12/06/2010 12:29 PM, Peter Soetens wrote:
>
> > You're probably worried that the
> > corba stuff might break when other transports aren't playing nice in the same
> > repository. But can't we easily solve this using cmake logic which only
> > builds a transport if the dependencies are present ?
> And why should I "behind the scenes" get the LUA scripting just because

Please, it is "Lua" which means moon in portugese, not some acronym.

Markus

Status for 2.2.0 release

On 12/06/2010 01:40 PM, Markus Klotzbuecher wrote:
> On Mon, Dec 06, 2010 at 01:31:13PM +0100, Sylvain Joyeux wrote:
>> On 12/06/2010 12:29 PM, Peter Soetens wrote:
>>
>>> You're probably worried that the
>>> corba stuff might break when other transports aren't playing nice in the same
>>> repository. But can't we easily solve this using cmake logic which only
>>> builds a transport if the dependencies are present ?
>> And why should I "behind the scenes" get the LUA scripting just because
>
> Please, it is "Lua" which means moon in portugese, not some acronym.
All my most abject apologies ;-)

/me does not like uppercase even for acronyms ...

Sylvain

Status for 2.2.0 release

On Mon, Dec 06, 2010 at 03:03:07PM +0100, Sylvain Joyeux wrote:
> On 12/06/2010 01:40 PM, Markus Klotzbuecher wrote:
> > On Mon, Dec 06, 2010 at 01:31:13PM +0100, Sylvain Joyeux wrote:
> >> On 12/06/2010 12:29 PM, Peter Soetens wrote:
> >>
> >>> You're probably worried that the
> >>> corba stuff might break when other transports aren't playing nice in the same
> >>> repository. But can't we easily solve this using cmake logic which only
> >>> builds a transport if the dependencies are present ?
> >> And why should I "behind the scenes" get the LUA scripting just because
> >
> > Please, it is "Lua" which means moon in portugese, not some acronym.
> All my most abject apologies ;-)
>
> /me does not like uppercase even for acronyms ...

You are forgiven this time...

:-)

Markus

Status for 2.2.0 release

On 06/12/2010 12:29, Peter Soetens wrote:
> On Monday 06 December 2010 12:16:29 Sylvain Joyeux wrote:
>> On 12/06/2010 11:56 AM, Peter Soetens wrote:
>>> I want to 'code freeze' today the repositories on orocos-toolchain. This
>>> means that each maintainer should merge the last patches and then update
>>> the version number of the package.
>> What does that mean in practice ?
> I'm updating rtt and ocl to '2.2.0'. The other libraries in the toolchain
> should increase their version number too to express 'major feature update',
> which you should do. I'll update log4cpp too to version '1.1'.
>
>>> There's also a 'transports' repository on gitorious that contains
>>> Charles' Yarp transport code. In the near future, I'd like to move the
>>> 'corba', 'ros' and 'mqueue' transports to there too. I won't release
>>> 'transports' yet with 2.2, but It should be ready for 2.3.
>> I would rather not build yet another "dump all" package (the "other"
>> being OCL). Could we move to a "one package per transport" policy ?
> With 'package' you mean 'git repository' ?
>
> I don't see transports as a 'dump all' thing. You're probably worried that the
> corba stuff might break when other transports aren't playing nice in the same
> repository. But can't we easily solve this using cmake logic which only
> builds a transport if the dependencies are present ?

Currently, I have added an option on the transports' root Cmake file to
enable building the Yarp transport. By keeping this organisation, we
should have a build command such as:
cmake .. -DENABLE_YARP=ON -DENABLE_CORBA=ON

The Yarp transport is still in beta stage, mainly because it still needs
more tests and documentation ; but works for my own simple usecase.

Charles.