RTT Flexiport branch

Hi,

I have just pushed a new branch to RTT:
https://gitorious.org/orocos-toolchain/rtt/commits/master-flexiport

It contains some modifications to the Connection chain that allows to
convert types for local connections, i.e., if you connect OutputPort<A> to
InputPort<B> in the same process, and provide a constructor B(A), then the
conversion will be done by the connection itself.

When I run ctest, it results in:
The following tests FAILED:
7 - corba-ipc-test (Failed)
35 - property_test (Failed)

As these tests also fail for me in the master branch, I guess that it is
not caused by my modifications.

Peter/Sylvain, I would be reassured if you look at the code to check that I
correctly manipulated ChannelElement and DataSource objects, especially
with respect to allocation...

I join a little test I made that connect ports with a conversion of simple
structs.

Seems to work ;)

Next step: extending to out-of-band connections (i.e. managing transports).

Charles.

AttachmentSize
flexiport_test.cpp2.02 KB

RTT Flexiport branch

Hi Charles,

On Wed, Oct 24, 2012 at 10:04 AM, Charles Lesire-Cabaniols
<charles [dot] lesire [..] ...> wrote:
> Hi,
>
> I have just pushed a new branch to RTT:
> https://gitorious.org/orocos-toolchain/rtt/commits/master-flexiport

Thanks ! Sorry for not responding sooner, I really had noticed :-)

>
> It contains some modifications to the Connection chain that allows to
> convert types for local connections, i.e., if you connect OutputPort<A> to
> InputPort<B> in the same process, and provide a constructor B(A), then the
> conversion will be done by the connection itself.
>
> When I run ctest, it results in:
> The following tests FAILED:
> 7 - corba-ipc-test (Failed)
> 35 - property_test (Failed)
>
> As these tests also fail for me in the master branch, I guess that it is not
> caused by my modifications.

They depend on some cmake setup (starting corba-ipc-server and copying
some xml/cpf files).

>
> Peter/Sylvain, I would be reassured if you look at the code to check that I
> correctly manipulated ChannelElement and DataSource objects, especially with
> respect to allocation...

We will

>
> I join a little test I made that connect ports with a conversion of simple
> structs.
>
> Seems to work ;)
>
> Next step: extending to out-of-band connections (i.e. managing transports).

Do you need any help here ?

Peter

RTT Flexiport branch

2012/12/13 Peter Soetens <peter [..] ...>

> Hi Charles,
>
> On Wed, Oct 24, 2012 at 10:04 AM, Charles Lesire-Cabaniols
> <charles [dot] lesire [..] ...> wrote:
> > Hi,
> >
> > I have just pushed a new branch to RTT:
> > https://gitorious.org/orocos-toolchain/rtt/commits/master-flexiport
>
> Thanks ! Sorry for not responding sooner, I really had noticed :-)
>
> >
> > It contains some modifications to the Connection chain that allows to
> > convert types for local connections, i.e., if you connect OutputPort<A>
> to
> > InputPort<B> in the same process, and provide a constructor B(A), then
> the
> > conversion will be done by the connection itself.
> >
> > When I run ctest, it results in:
> > The following tests FAILED:
> > 7 - corba-ipc-test (Failed)
> > 35 - property_test (Failed)
> >
> > As these tests also fail for me in the master branch, I guess that it is
> not
> > caused by my modifications.
>
> They depend on some cmake setup (starting corba-ipc-server and copying
> some xml/cpf files).
>
> >
> > Peter/Sylvain, I would be reassured if you look at the code to check
> that I
> > correctly manipulated ChannelElement and DataSource objects, especially
> with
> > respect to allocation...
>
> We will
>

Good. Looking forward to reading from you

>
> >
> > I join a little test I made that connect ports with a conversion of
> simple
> > structs.
> >
> > Seems to work ;)
> >
> > Next step: extending to out-of-band connections (i.e. managing
> transports).
>
> Do you need any help here ?
>

I don't think so... but I just have no time for the moment. I will try to
look again into it by mid-January.

>
> Peter
>

RTT Flexiport branch

2012/12/14 Charles Lesire-Cabaniols <charles [dot] lesire [..] ...>

>
>
>
> 2012/12/13 Peter Soetens <peter [..] ...>
>
>> Hi Charles,
>>
>> On Wed, Oct 24, 2012 at 10:04 AM, Charles Lesire-Cabaniols
>> <charles [dot] lesire [..] ...> wrote:
>> > Hi,
>> >
>> > I have just pushed a new branch to RTT:
>> > https://gitorious.org/orocos-toolchain/rtt/commits/master-flexiport
>>
>> Thanks ! Sorry for not responding sooner, I really had noticed :-)
>>
>> >
>> > It contains some modifications to the Connection chain that allows to
>> > convert types for local connections, i.e., if you connect OutputPort<A>
>> to
>> > InputPort<B> in the same process, and provide a constructor B(A), then
>> the
>> > conversion will be done by the connection itself.
>> >
>> > When I run ctest, it results in:
>> > The following tests FAILED:
>> > 7 - corba-ipc-test (Failed)
>> > 35 - property_test (Failed)
>> >
>> > As these tests also fail for me in the master branch, I guess that it
>> is not
>> > caused by my modifications.
>>
>> They depend on some cmake setup (starting corba-ipc-server and copying
>> some xml/cpf files).
>>
>> >
>> > Peter/Sylvain, I would be reassured if you look at the code to check
>> that I
>> > correctly manipulated ChannelElement and DataSource objects, especially
>> with
>> > respect to allocation...
>>
>> We will
>>
>
> Good. Looking forward to reading from you
>
>
>>
>> >
>> > I join a little test I made that connect ports with a conversion of
>> simple
>> > structs.
>> >
>> > Seems to work ;)
>> >
>> > Next step: extending to out-of-band connections (i.e. managing
>> transports).
>>
>> Do you need any help here ?
>>
>
> I don't think so... but I just have no time for the moment. I will try to
> look again into it by mid-January.
>
>
Hi Peter (and others),

I just came back into the flexiport branch.

In ConnFactory::createConnection, you added the case where the input_port
is not local, but I can't see why and when we could actually lead to this
situation. My intuition is that when a port is not local, we should have
called createStream instead of createConnection.

Why did you consider this case? (the behavior in this case is not fully
implemented, so I guess it is a specific flexiport use case)
Is there a corner case I haven't seen?

A complementary question about connections vs. streams: I think it would
make sense to consider that creating an OutOfBand connection is just like
creating two streams, respectively for the input and output parts, wouldn't
it?

Charles.

>
>
>>
>> Peter
>>
>
>

RTT Flexiport branch

Hi Charles,

On Thu, Jan 10, 2013 at 8:31 AM, Charles Lesire-Cabaniols <
charles [dot] lesire [..] ...> wrote:

>
> Hi Peter (and others),
>
> I just came back into the flexiport branch.
>
> In ConnFactory::createConnection, you added the case where the input_port
> is not local, but I can't see why and when we could actually lead to this
> situation. My intuition is that when a port is not local, we should have
> called createStream instead of createConnection.
>

No... you could use createconnection on an output port of a regular TC and
an input port of a proxy TC and use the CORBA transport. The latter port
would then be in the case you describe above. It will be of type
'RemotePort', so isLocal() will return false. createStream is not
implemented by the CORBA transport, afaikt.

>
> Why did you consider this case? (the behavior in this case is not fully
> implemented, so I guess it is a specific flexiport use case)
> Is there a corner case I haven't seen?
>
> A complementary question about connections vs. streams: I think it would
> make sense to consider that creating an OutOfBand connection is just like
> creating two streams, respectively for the input and output parts, wouldn't
> it?
>

Yes, apart from the lifetime difference in both cases:

The 2 'streams' created by createStream are completely independent of each
other in terms of lifetime. Connections, outofband or not, are always a
single entity which are destructed/created in whole. A connection using an
outofband stream behind the scenes will also cleanup these 2 streams when
the connection is destroyed.

If you apply this to the mqueue transport, you can use CORBA to connect two
ports of different processes on the same machine using the outofband mqueue
transport and manage the mqueue lifetimes using the connection object. I'm
quite sure this is the specific use case why outofband exists.

Peter

RTT Flexiport branch

2013/1/11 Peter Soetens <peter [..] ...>

> Hi Charles,
>
> On Thu, Jan 10, 2013 at 8:31 AM, Charles Lesire-Cabaniols <
> charles [dot] lesire [..] ...> wrote:
>
>>
>> Hi Peter (and others),
>>
>> I just came back into the flexiport branch.
>>
>> In ConnFactory::createConnection, you added the case where the input_port
>> is not local, but I can't see why and when we could actually lead to this
>> situation. My intuition is that when a port is not local, we should have
>> called createStream instead of createConnection.
>>
>
> No... you could use createconnection on an output port of a regular TC and
> an input port of a proxy TC and use the CORBA transport. The latter port
> would then be in the case you describe above. It will be of type
> 'RemotePort', so isLocal() will return false. createStream is not
> implemented by the CORBA transport, afaikt.
>
>

>
>>
>> Why did you consider this case? (the behavior in this case is not fully
>> implemented, so I guess it is a specific flexiport use case)
>> Is there a corner case I haven't seen?
>>
>> A complementary question about connections vs. streams: I think it would
>> make sense to consider that creating an OutOfBand connection is just like
>> creating two streams, respectively for the input and output parts, wouldn't
>> it?
>>
>
> Yes, apart from the lifetime difference in both cases:
>
> The 2 'streams' created by createStream are completely independent of each
> other in terms of lifetime. Connections, outofband or not, are always a
> single entity which are destructed/created in whole. A connection using an
> outofband stream behind the scenes will also cleanup these 2 streams when
> the connection is destroyed.
>
> If you apply this to the mqueue transport, you can use CORBA to connect
> two ports of different processes on the same machine using the outofband
> mqueue transport and manage the mqueue lifetimes using the connection
> object. I'm quite sure this is the specific use case why outofband exists.
>

I am not sure to have fully understood your statements.

What is not clear in
http://www.orocos.org/stable/documentation/rtt/v2.x/doc-xml/orocos-trans...
whether the first example (with TaskContexts) is using CORBA or not...
looks like CORBA is not used in this case, so that a pure MQueue OOB
connection is created!

If I use a CORBA proxy, I will be in the Remote case, not in the OOB, right?

Do you have a test case and an example of a Remote (using CORBA) connection
so that I could test the flexiport branch on it?

I have just pushed the createStream parts, that work with MQueue on my side
(in both output and input sides). So Remote and OOB are the only things to
fix now.

Charles.

>
> Peter
>
>

RTT Flexiport branch

Charles,

On Mon, Jan 21, 2013 at 10:33 AM, Charles Lesire-Cabaniols
<charles [dot] lesire [..] ...> wrote:
>
>
>
> 2013/1/11 Peter Soetens <peter [..] ...>
>>
>> Hi Charles,
>>
>> On Thu, Jan 10, 2013 at 8:31 AM, Charles Lesire-Cabaniols
>> <charles [dot] lesire [..] ...> wrote:
>>>
>>>
>>> Hi Peter (and others),
>>>
>>> I just came back into the flexiport branch.
>>>
>>> In ConnFactory::createConnection, you added the case where the input_port
>>> is not local, but I can't see why and when we could actually lead to this
>>> situation. My intuition is that when a port is not local, we should have
>>> called createStream instead of createConnection.
>>
>>
>> No... you could use createconnection on an output port of a regular TC and
>> an input port of a proxy TC and use the CORBA transport. The latter port
>> would then be in the case you describe above. It will be of type
>> 'RemotePort', so isLocal() will return false. createStream is not
>> implemented by the CORBA transport, afaikt.
>>
>>
>>
>>>
>>>
>>> Why did you consider this case? (the behavior in this case is not fully
>>> implemented, so I guess it is a specific flexiport use case)
>>> Is there a corner case I haven't seen?
>>>
>>> A complementary question about connections vs. streams: I think it would
>>> make sense to consider that creating an OutOfBand connection is just like
>>> creating two streams, respectively for the input and output parts, wouldn't
>>> it?
>>
>>
>> Yes, apart from the lifetime difference in both cases:
>>
>> The 2 'streams' created by createStream are completely independent of each
>> other in terms of lifetime. Connections, outofband or not, are always a
>> single entity which are destructed/created in whole. A connection using an
>> outofband stream behind the scenes will also cleanup these 2 streams when
>> the connection is destroyed.
>>
>> If you apply this to the mqueue transport, you can use CORBA to connect
>> two ports of different processes on the same machine using the outofband
>> mqueue transport and manage the mqueue lifetimes using the connection
>> object. I'm quite sure this is the specific use case why outofband exists.
>
>
> I am not sure to have fully understood your statements.
>
> What is not clear in
> http://www.orocos.org/stable/documentation/rtt/v2.x/doc-xml/orocos-trans...
> in whether the first example (with TaskContexts) is using CORBA or not...
> looks like CORBA is not used in this case, so that a pure MQueue OOB
> connection is created!

Yes, it's a pure mqueue connection, but it's not out-of-band ! there
is only one band, and that is mqueue.

So pure mqueue looks like this:

outport -> endpoint ->mqueue write -> | -> mqueue read -> endpoint > inport

This is very similar to CORBA, ROS etc.

And the OOB looks like this:

outport -> endpoint -> corba chan -> mqueue write -> | -> mqueue read
-> corba chan -> endpoint -> inport

I'm abstracting the use of a buffer on either side here...

so in 'true' OOB, the corba channel element passes the data, signal
etc on to the mqueue channel element.

That's why the CORBA code in RemoteChannelElement.hpp first tries to
use the base class and if that
fails, uses the CORBA pathway. So 'corba chan' needs to be aware of
out-of-band situations.

>
> If I use a CORBA proxy, I will be in the Remote case, not in the OOB, right?

If you use a CORBA proxy and then select the ORO_MQUEUE_PROTOCOL_ID in
your ConnPolicy,
it will first setup a CORBA connection and then in there the mqueue elements.

>
> Do you have a test case and an example of a Remote (using CORBA) connection
> so that I could test the flexiport branch on it?

The unit tests cover these cases afaikt.

>
> I have just pushed the createStream parts, that work with MQueue on my side
> (in both output and input sides). So Remote and OOB are the only things to
> fix now.

I'll have to delay a review/test a bit, but I'm not forgetting about it :-)

Peter