It's been FOSDEM last weekend and as is every year, this two-day conference
was a huge source of inspiration and reflection. Listening to what other people
do and need offers many opportunities for cooperation. And that's exactly what
this project is for. So naturally, I wondered how other projects could benefit
from the RTT.
There was this guy with a board with an ARM and an Ethernet port. Assuming
that his system was so embedded that no more than one Orocos component needed
to run on it, I wondered how 'junior' would then communicate with the
'server'. He couldn't use some shell script to remotely toggle a property or
to log some data off a data port (we don't got any). He couldn't even implement
the protocol the RTT uses (remember, its CORBA) or needed dragging in huge
libraries. Finally I wondered how usable the RTT is in such a setup. Well that
is if we got that far explaining the poor fella the intricacies of the RTT
component interface.
People knowing me know that after such a case, something better gets proposed
next. But I'd like to point out the pain points I was touching here:
* RTT interoperates badly with other software, for example, any external
process needs to go through a convoluted CORBA layer. There are also no tools
that could ease the job (except the ctaskbrowser), for example some small
shell commands that can query/change a component.
* RTT has remaining usability issues. Sylvain already identified the short
commings of data/buffer ports and proposed a solution. But any user wrestling
with the ''Should I use an Event (syn/asyn)-Method-Command-DataPort'?'
question only got the answer: ''Well, we got Events(syn/asyn)-Methods-
Commands and DataPorts !'. It's not coherent. There are other frameworks doing
a better job. We can do a far better job.
* RTT has issues with its current distribution implementation: programs can be
constructed as such that they cause mem leaks at the remote side, Events never
got into the CORBA interface (there is a reason for that), and our data ports
over CORBA are equally weak as the C++ implementation.
* And then there are also the untaken opportunities to reduce RTT & component
code size drastically and remove complex features.
What does this all mean ? Start over ? Not at all. I'll defend that with small
changes (of which some break backwards compatibility) we can solve all the
above in one swift sweep. Unfortunately, it all starts with adding a new (but
optionally optional) feature: rt_malloc, aka TLSF. Yes you saw me write I
would remove features, by adding a feature (well, actually two). I can
explain.
I've seen people using the RTT for inter-thread communication in two major
ways: or implement a function as a Method, or as a Command. Where the command
was the thread-safe way to change the state of a component. The adventurous
used Events as well, but I can't say they're a huge success (we got like only
one 'thank you' email in its whole existence...). But anyway, Commands are
complex for newbies, Events (syn/asyn) aren't better. So for all these people,
here it comes: the RTT::Message object. Remember, Methods allow a peer
component to _call_ a function foo(args) of the component interface. Messages
will have the meaning of _sending_ another component a message to execute a
function foo(args). Contrary to Methods, Messages are 'send and forget', they
return void. The only guarantee you got is, that if the receiver was active,
it processed it. For now, forget that Commands exist. We have two inter-
component messaging primitives now: Messages and Methods. And each component
declares: You can call these methods and send these messages.
They are the 'Level 0' primitives of the RTT. Any transport should support
these. Note that conveniently, the transport layer may implement messages with
the same primitive as data ports. But we, users, don't care. We still have
Data Ports to 'broadcast' our data streams and now we have Messages as well to
send directly to component X.
Think about it. The RTT would be highly usable if each component only had data
ports and a Message/Method interface. Ask the AUTOSAR people, it's very close
to what they have (and can live with).
There's one side effect of the Message: we will need a real-time memory
allocator to reserve a piece of memory for each message sent, and free it when
the message is processed. Welcome TLSF. In case such a thing is not possible
wanted by the user, Messages can fall back to using pre-allocated memory, but
at the cost of reduced functionality (similar to what Commands can do today).
Also, we'll have a MessageProcessor, which replaces and is a slimmed down
version of the CommandProcessor today.
So where does this leave Events? Events are of the last primitives I explain
in courses because they are so complex. They don't need to be. Today you need
to attach a C/C++ function to an event and optionally specify an
EventProcessor. Depending on some this-or-thats the function is executed in
this-or-the-other thread. Let's forget about that. In essence, an Event is a
local thing that others like to know about: Something happened 'here', who
wants to know ? Events can be changed such that you can say: If event 'e'
happens, then call this Method. And you can say: if event 'e' happens, send me
this Message. You can subscribe as many callbacks as you want.
Because of the lack of this mechanism, the current Event implementation has a
huge foot print. There's a lot to win here.
Do you want to allow others to raise the event ? Easy: add it to the Message
or Method interface, saying: send me this Message and I'll raise the event, or
call this Method and you'll raise it, respectively. But if someone can raise
it, is your component's choice. That's what the event interface should look
like. It's a Level 1. A transport should do no more than allowing to connect
Methods and Messages (which it already supports, Level 1) to Events. No more.
Even our CORBA layer could do that.
The implementation of Event can benefit from a rt_malloc as well. Indirectly.
Each raised Event which causes Messages to be sent out will use the Message's
rt_malloc to store the event data by just sending the Message. In case you
don't have/want an rt_malloc, you fall back to what events can roughly do
today. But with lots of less code ( Goodbye RTT::ConnectionC, Goodbye
RTT::EventProcessor ).
And now comes the climax: Sir Command. How does he fit in the picture? He'll
remain in some form, but mainly as a 'Level 2' citizen. He'll be composed of
Methods, Messages and Events and will be dressed out to be no more than a
wrapper, keeping related classes together or even not that. Replacing a
Command with a Message hardly changes anything in the C++ side. For scripts,
Commands were damn useful, but we will come up with something satisfactory.
I'm sure.
How does all this interface shuffling allows us to get 'towards a sustainable
distributed component model'? That's because we're seriously lowering the
requirements on the transport layer:
* It only needs to implement the Level 0 primitives. How proxies and servers
are built depends on the transport. You can do so manually (dlib like) or
automatically (CORBA like)
* It allows the transport to control memory better, share it between clients
and clean it up at about any time.
* The data flow changes Sylvain proposes strengthen our data flow model and I'm
betting on it that it won't use CORBA as a transport. Who knows.
And we are at the same time lowering the learning curve for new users:
* You can easily explain the basic primitives: Properties=>XML,
DataPorts=>process data, Methods/Messages=>client/server requests. When
they're familiar with these, they can start playing with Events (which build
on top of Method/Messages and play a role in DataPorts as well). And finally,
if they'll ever need, the Convoluted Command can encompass the most complex
scenarios.
* You can more easily connect with other middleware or external programs.
People with other middleware will see the opportunities for 1-to-1 mappings or
even implement it as a transport in the RTT.
So this is my plan for RTT 2.0(09): not UML state machines, not generated
code, not lua integration. They can wait for 3.0. Cleaning up the interface,
unifying it, providing all users a smooth upgrade path, being similar to what
other component frameworks offer. Setting a path out for growth. That's 2.0.
These major steps won't keep us from improving what we have, There's a lot
we're keeping the same. I'll setup a RTT 2.0 branch with git. By next FOSDEM,
I'd like to demonstrate it. In 45 minutes. With a small ARM board. Running one
component. which I conveniently control from my command line, web interface or
taskbrowser on my Laptop. Or any other laptop in the audience.
It'll be fun.
Peter
The road to RTT 2.0: towards a sustainable distributed
First, let me say that this discussion on the Orocos 2.0 roadmap is very welcome.
I will give my opinion as user of Orocos, but also as scientist which is very concerning about the huge number of middleware out there.
Orocos has some very good characteristics in my opinion and some faults, which Peter seems to recognize.
It is probably one of the few middleware targeting robotics who takes into account real-time and present a structure to write its components which almost resemble a "design pattern" to write robotics components.
The bad point is that:
1) the learning curve is not very good. More example could really help, and also a better naming of the functions.
2) I was going to say that we should get rid of all this Method+Commands+Events stuff, but Herman did actually changed my mind when he says that:
"you will immediately see reinvention of the "Method", "Event" and "Command" primitives, but done poorly, and in incompatible ways..."
3) the footprint of Orocos (and may be the number of dependencies) is a serious problem for the future of the Orocos project, in my opinion. And CORBA is a fat monster! Peter says that is one of the implementation but not the only one. I would suggest to follow the direction of WilloW Garage and ROS, where a slim TCP/UDP/shared memory communication is chosen according to the way that components need to communicate.
4) I think that several threads running on a single process is not the most easy to use and robust way to build a robotic framework. We should forget the Deployer and use SLIM and efficient inter-process communication (one more time, I suggest the ROS approach).
5) 1-to-1 mapping to other robotic framework IS the key to get more people into Orocos and re-use software. On the other hand, I would suggest to learn from other projects and collaborate with them to create just ONE framework with takes the best of the middleware family (did does not means that it need to be big and complex... actually, I think the best solution is to keep it as much scalable as possible).
Let me say that we really appreciate the work done on Orocos and I eagerly want your software to be used by as many people as possible! This is the reason why we NEED to carefully decide the roadmap to be taken.
Regards
Davide
The road to RTT 2.0: towards a sustainable distributed
On Fri, 20 Feb 2009, faconti [..] ... wrote:
[...]
> 5) 1-to-1 mapping to other robotic framework IS the key to get more
> people into Orocos and re-use software. On the other hand, I would
> suggest to learn from other projects and collaborate with them to create
> just ONE framework with takes the best of the middleware family (did does
> not means that it need to be big and complex... actually, I think the
> best solution is to keep it as much scalable as possible).
>
I want to come back to this particular remark... Since the conception of
Orocos, we had this 'design requirement' in mind, and in a prominent place
even. Our answer to this was and is to provide functionalities in small
chunks, with APIs and classes that really belong together without the
possibility to make the library smaller. This is a rather different
approach than most projects, who put too many couplings between classes and
concepts that could be separated, because they are too much driven by one
single application. I think this holds true for Willow Garage also: I have
never read any information from them that they want (parts of) their
software to be usable in machine tools, or satellites, or space robots...
Orocos _does_ have this ambition, and we are working towards it at the
expense of a lot of configuration options and many small libraries. (We are
not yet there, since I know we can decouple already quite some of the code
we have in smaller libraries...)
The long term vision in this context is to develop a toolchain that
supports the users/application builders in working with this "configuration
hell" in a decent way; this "long term vision" is becoming a "medium
term" vision, because it is the ambition of the BRICS project :-) Stay
tuned :-)
BTW, I am still looking for a full time developer to take care of that
toolchain support! If you are interested, available for four years, and
experienced in Eclipse and its Rich Client Platform (or motivated enough to
become an expert), please give me a call!!!! :-)
Herman
The road to RTT 2.0: towards a sustainable distributed
On Feb 20, 2009, at 12:41 , faconti [..] ... wrote:
> First, let me say that this discussion on the Orocos 2.0 roadmap is
> very welcome.
>
> I will give my opinion as user of Orocos, but also as scientist
> which is very concerning about the huge number of middleware out
> there.
My turn for opinions ...
> Orocos has some very good characteristics in my opinion and some
> faults, which Peter seems to recognize.
>
> It is probably one of the few middleware targeting robotics who
> takes into account real-time and present a structure to write its
> components which almost resemble a "design pattern" to write
> robotics components.
>
> The bad point is that:
>
> 1) the learning curve is not very good. More example could really
> help, and also a better naming of the functions.
Very very true. IMHO this is *the* single biggest issue with Orocos. I
want to change this ...
> 2) I was going to say that we should get rid of all this Method
> +Commands+Events stuff, but Herman did actually changed my mind when
> he says that:
>
> "you will immediately see reinvention of the "Method", "Event" and
> "Command" primitives, but done poorly, and in
> incompatible ways..."
I think perhaps some consolidation of Methods/Commands is warranted.
On one of our projects we use about 60 methods, 0 commands, plus a
handful of scripting functions (which are realised as commands, I
believe).
> 3) the footprint of Orocos (and may be the number of dependencies)
> is a serious problem for the future of the Orocos project, in my
> opinion. And CORBA is a fat monster! Peter says that is one of the
> implementation but not the only one. I would suggest to follow the
> direction of WilloW Garage and ROS, where a slim TCP/UDP/shared
> memory communication is chosen according to the way that components
> need to communicate.
The footprint of Orocos is fine with me, though Corba is horrid I'll
admit. We use Orocos on a range of embedded and desktop machines,
ranging from 1GHz CPU and up (so admittedly, moderate sized machines).
We've had zero trouble with any footrpint aspect like CPU performance,
RAM usage, etc.
I personally do *not* want to go to yet another custom-brewed IP
communications strategy. I *want* to use middleware for this also,
especially the name-and-forget publish/subscribe model offered by
Corba and its brethren.
> 4) I think that several threads running on a single process is not
> the most easy to use and robust way to build a robotic framework. We
> should forget the Deployer and use SLIM and efficient inter-process
> communication (one more time, I suggest the ROS approach).
I diasgree with a blanket statement like this. This single process
model is a very efficient model, and often (though is not a necessity)
goes hand in hand with the real-time nature that you saluted Orocos
for in the top of the email. It can also lend itself to much easier
deployment on very small embedded systems. Currently you can actually
do this with Orocos, as you could deploy one component per deployer
and have the communications middleware connect them all. Definitely
not efficient, but it shows that this is possible with the current
framework.
Personally, I *love* the deployer and would argue mightily against
having it removed. Also, the deployer does more than just start a
process with several components/threads running.
> 5) 1-to-1 mapping to other robotic framework IS the key to get more
> people into Orocos and re-use software. On the other hand, I would
> suggest to learn from other projects and collaborate with them to
> create just ONE framework with takes the best of the middleware
> family (did does not means that it need to be big and complex...
> actually, I think the best solution is to keep it as much scalable
> as possible).
That sounds nice, having one framework/solution that fits all, but
I've never ever seen this happen. Competition is a good thing also ....
> Let me say that we really appreciate the work done on Orocos and I
> eagerly want your software to be used by as many people as possible!
> This is the reason why we NEED to carefully decide the roadmap to be
> taken.
Agreed. It's a wonderful framework with many good and bad points to
it. This roadmap should prompt discussion on how to improve Orocos.
Cheers
Stephen
The road to RTT 2.0: towards a sustainable distributed
On Fri, 20 Feb 2009, S Roderick wrote:
> On Feb 20, 2009, at 12:41 , faconti [..] ... wrote:
>
>> First, let me say that this discussion on the Orocos 2.0 roadmap is
>> very welcome.
>>
>> I will give my opinion as user of Orocos, but also as scientist
>> which is very concerning about the huge number of middleware out
>> there.
>
> My turn for opinions ...
>
>> Orocos has some very good characteristics in my opinion and some
>> faults, which Peter seems to recognize.
>>
>> It is probably one of the few middleware targeting robotics who
>> takes into account real-time and present a structure to write its
>> components which almost resemble a "design pattern" to write
>> robotics components.
>>
>> The bad point is that:
>>
>> 1) the learning curve is not very good. More example could really
>> help, and also a better naming of the functions.
>
> Very very true. IMHO this is *the* single biggest issue with Orocos. I
> want to change this ...
Most of us want to help out in one way or another! :-)
>> 2) I was going to say that we should get rid of all this Method
>> +Commands+Events stuff, but Herman did actually changed my mind when
>> he says that:
>>
>> "you will immediately see reinvention of the "Method", "Event" and
>> "Command" primitives, but done poorly, and in
>> incompatible ways..."
>
> I think perhaps some consolidation of Methods/Commands is warranted.
> On one of our projects we use about 60 methods, 0 commands, plus a
> handful of scripting functions (which are realised as commands, I
> believe).
What is your definition of "consolidation"?
>> 3) the footprint of Orocos (and may be the number of dependencies)
>> is a serious problem for the future of the Orocos project, in my
>> opinion. And CORBA is a fat monster! Peter says that is one of the
>> implementation but not the only one. I would suggest to follow the
>> direction of WilloW Garage and ROS, where a slim TCP/UDP/shared
>> memory communication is chosen according to the way that components
>> need to communicate.
>
> The footprint of Orocos is fine with me, though Corba is horrid I'll
> admit.
How many times do we have to repeat that CORBA is _not_ a built-in
dependency! If you like slim TCP/UPD/shared memory/... communication,
please provide the glue to use that to replace CORBA...
Again, although nobody really likes CORBA, a similar remark as with the
Commands holds here: Willow Garage _will_ reinvent many of the CORBA
concepts, because (many of) these concepts are good...
In addition, Orocos' design is (more or less) ready for automatic code
generation, including translation into the smalles footprint that is
possible _if_ your hardware system is fully centralized. But also in this
respect we need discussions and contributions.
> We use Orocos on a range of embedded and desktop machines,
> ranging from 1GHz CPU and up (so admittedly, moderate sized machines).
> We've had zero trouble with any footrpint aspect like CPU performance,
> RAM usage, etc.
>
> I personally do *not* want to go to yet another custom-brewed IP
> communications strategy. I *want* to use middleware for this also,
> especially the name-and-forget publish/subscribe model offered by
> Corba and its brethren.
>
>> 4) I think that several threads running on a single process is not
>> the most easy to use and robust way to build a robotic framework. We
>> should forget the Deployer and use SLIM and efficient inter-process
>> communication (one more time, I suggest the ROS approach).
> I diasgree with a blanket statement like this. This single process
> model is a very efficient model, and often (though is not a necessity)
> goes hand in hand with the real-time nature that you saluted Orocos
> for in the top of the email. It can also lend itself to much easier
> deployment on very small embedded systems. Currently you can actually
> do this with Orocos, as you could deploy one component per deployer
> and have the communications middleware connect them all. Definitely
> not efficient, but it shows that this is possible with the current
> framework.
As S mentions above, Orocos _never_ forces you to use multiple threads...
It offers this functionality since it is often very useful to have in
realtime contexts. But I also do agree with you that we need some well
documented examples of applying Orocos in the "Willow Garage use case" kind
of applications. (However, we already _have_ such examples, such as the
"open house demo" which has a video on the website: five different
computers are involved, so that _are_ five different processes, at least.
using CORBA there would not have been mandatory, but we didn't have to
spend time writing the TCP-based IPC...)
> Personally, I *love* the deployer and would argue mightily against
> having it removed. Also, the deployer does more than just start a
> process with several components/threads running.
The Deployer is indeed one of the core functionalities of Orocos, and one
of the largest differentiator with respect to other frameworks. And it will
get more improvements in the future, undoubtedly, because robotic systems
get more complex all the time... I do hope we can switch to an appropriate
external framework for deployment also in the near future, and OSGi is a
very good and reasonably mature and appropriate candidate. Of course, this
would again import a lot of "bloat" and dependencies, but it will be _much_
better and cheaper than redoing everything ourselves again.
>> 5) 1-to-1 mapping to other robotic framework IS the key to get more
>> people into Orocos and re-use software. On the other hand, I would
>> suggest to learn from other projects and collaborate with them to
>> create just ONE framework with takes the best of the middleware
>> family (did does not means that it need to be big and complex...
>> actually, I think the best solution is to keep it as much scalable
>> as possible).
>
> That sounds nice, having one framework/solution that fits all, but
> I've never ever seen this happen. Competition is a good thing also ....
Yes, and Orocos was first! :-) So, please make the same remarks to the ROS
people :-)
>> Let me say that we really appreciate the work done on Orocos and I
>> eagerly want your software to be used by as many people as possible!
>> This is the reason why we NEED to carefully decide the roadmap to be
>> taken.
>
> Agreed. It's a wonderful framework with many good and bad points to
> it. This roadmap should prompt discussion on how to improve Orocos.
I welcome all your critical remarks _a lot_, because the worst thing we
could do is to ignore the efforts of people like you who went through the
"pain" to give Orocos a fair try! Nevertheless, Orocos _has_ a long-term
vision (which I think is much more of a vision than the "competitors"...)
and we know very much that we are not there yet... Just to make sure that
we understand each other well: this vision is to provide a framework (and
more and more functionalities to put into this framework's "hot spots")
that can support the most professional and complex robotic and machine tool
control systems of the future. And, by the way, one single humanoid robot
the way PAL robotics makes them is a _simple_ system in Orocos'
context...:-) The applications we are targeting will be three of such
humanoids, working together in _physical_ contact with the same objects,
while standing on a moving catamaran :-)
In summary: I'm all for slim implementations of some of Orocos' hot spots,
but not for giving up our "heavy" component- and service-based vision :-)
Herman
The road to RTT 2.0: towards a sustainable distributed component
On Mon, Feb 9, 2009 at 12:26 AM, Peter Soetens <peter [dot] soetens [..] ...> wrote:
> It's been FOSDEM last weekend and as is every year, this two-day conference
> was a huge source of inspiration and reflection. Listening to what other people
> do and need offers many opportunities for cooperation. And that's exactly what
> this project is for. So naturally, I wondered how other projects could benefit
> from the RTT.
>
> There was this guy with a board with an ARM and an Ethernet port. Assuming
> that his system was so embedded that no more than one Orocos component needed
> to run on it, I wondered how 'junior' would then communicate with the
> 'server'. He couldn't use some shell script to remotely toggle a property or
> to log some data off a data port (we don't got any). He couldn't even implement
> the protocol the RTT uses (remember, its CORBA) or needed dragging in huge
> libraries. Finally I wondered how usable the RTT is in such a setup. Well that
> is if we got that far explaining the poor fella the intricacies of the RTT
> component interface.
As a side note, I somehow fail to see the relation between the FOSDEM
conf and the rest of your email? We have seen guys with arm boards
with and without ethernet ports before right? AutoSAR on FOSDEM? Most
of the things below were (to my surprise) not really knew to me
(unless I misunderstood them :-) and I felt like we discussed them
before. Ofcourse, this makes it harder for me to disagree :-)
This said...
[...]
> Think about it. The RTT would be highly usable if each component only had data
> ports and a Message/Method interface. Ask the AUTOSAR people, it's very close
> to what they have (and can live with).
[Omit the implementation part about the RTmalloc for now in the API discussion]
> So where does this leave Events? Events are of the last primitives I explain
> in courses because they are so complex. They don't need to be. Today you need
> to attach a C/C++ function to an event and optionally specify an
> EventProcessor. Depending on some this-or-thats the function is executed in
> this-or-the-other thread. Let's forget about that. In essence, an Event is a
> local thing that others like to know about: Something happened 'here', who
> wants to know ? Events can be changed such that you can say: If event 'e'
> happens, then call this Method. And you can say: if event 'e' happens, send me
> this Message. You can subscribe as many callbacks as you want.
> Because of the lack of this mechanism, the current Event implementation has a
> huge foot print. There's a lot to win here.
>
> Do you want to allow others to raise the event ? Easy: add it to the Message
> or Method interface, saying: send me this Message and I'll raise the event, or
> call this Method and you'll raise it, respectively. But if someone can raise
> it, is your component's choice. That's what the event interface should look
> like. It's a Level 1. A transport should do no more than allowing to connect
> Methods and Messages (which it already supports, Level 1) to Events. No more.
> Even our CORBA layer could do that.
Enter UML spec:
In UML, the "UML::Message" is the base primitive for interObject
(think Sequence Diagram) communication. Apart from some exotic
constructions, there are 3 "kinds" of Messages (see e.g. Fig 14.5 of
the latest spec),
- synchCall (mapping to what you call Method above)
- asynchCall (mapping to what you call RTT::Message above)
- asynchSignal (mapping "plus minus" to RTT::Event above)
So:
1/ Since we are moving in a direction compliant with the spec, I like
the above suggestions
2/ However, it might be a good time to revise the naming? Was there
anyone who intuitively understood the fact that a RTT::Method was
something synchronous and the (now level 2) RTT::Command something
asynchronous?
Personnally, I think the UML names are more speaking-for-themselves
than your suggestions of Message/Method (although that does not take
into account our legacy users though)
3/ [I saved the most complex for last :]
RTT::Events. I would describe this in a metaphor (heck I didn't even
bother to check whether this is a correct englisch word) as a
component (UML::Object) raising a "flag", so that others looking at
the component can react appropriately (i.e. a primitive for _inter_
object communication).
Entering Stephen commments about "Hey, I find them (rtt::events) very
useful, especially in my FSMs". Exactly. However, FSM describes the
behaviour of a certain component and not the inter-component
communication. In UML, one has
- Events (something happened):
- Signals (interobject _communication_)
There is clearly a relationship between this (UML::SignalEvent), but
In orocos 1.x (and in your current proposal), I have the gut feeling
these 2 things are somehow mixed up (I dont' know whether this is
necessarily a bad thing, or a way of implementing/interpreting the UML
spec.)
E.g. when we need a TimeEvent in our orocos FSM, we rely on a 3rd
component that delivers us a UML::Signal.
Maybe we should think wether the Event/Signal separation could also be
useful in RTT 2.0
Klaas (going back to sleep now :-)
The road to RTT 2.0: towards a sustainable distributed component
On Mon, Feb 16, 2009 at 5:53 PM, Sylvain Joyeux <sylvain [dot] joyeux [..] ...> wrote:
>> Can you rephrase/elaborate that question? I don't understand what you
>> mean with "the need *between*" in the sentence above.
> Sorry, missing some words.
>
> What I meant, is "the need to have a distinction between an event and a
> signal". In other words: what are *exactly* events and signals in UML terms ?
[Disclaimer: I should _really_ check what I say below
Also see my original reply at <http://www.orocos.org/node/1068#comment-3103> ]
If I have it right, an UML::Signal is an *inter-object communication
primitive* (a particular kind of message), a callback mechanism. One
object registers a callback (handler), and when the sender "signals",
all callbacks are dealt with by those who have registered.
An UML::event is something that *occurs*, but which is not
(necessarily?) related to inter-object communication. Hence, in
statemachines, one can trigger a transition based on the occurence of
a time-event (e.g. 10 secs have elapsed), or a call-event, ...
As I said in my original post, it ain't 100% clear to me either :(.
But I think that in orocos currently everything is expressed as a
signal (hence a mapping between RTT::Event and UML::Signal)
E.g. RTT::TimeEvents are currently expressed as Signals (i.e.
RTT::Events emitted by a separate component --- TimerComponent) .
So I wondered wether maybe the Event/Signal distinction could be
useful in orocos too, e.g. to simplify the use of TimeEvents in the
StateMachine specifications.
HTH (somewhat :-)
Klaas
The road to RTT 2.0: towards a sustainable distributed component
Hi Klaas,
On Mon, Feb 16, 2009 at 06:21:50PM +0100, Klaas Gadeyne wrote:
> On Mon, Feb 16, 2009 at 5:53 PM, Sylvain Joyeux <sylvain [dot] joyeux [..] ...> wrote:
> >> Can you rephrase/elaborate that question? I don't understand what you
> >> mean with "the need *between*" in the sentence above.
> > Sorry, missing some words.
> >
> > What I meant, is "the need to have a distinction between an event and a
> > signal". In other words: what are *exactly* events and signals in UML terms ?
>
> [Disclaimer: I should _really_ check what I say below
> Also see my original reply at <http://www.orocos.org/node/1068#comment-3103> ]
>
> If I have it right, an UML::Signal is an *inter-object communication
> primitive* (a particular kind of message), a callback mechanism. One
> object registers a callback (handler), and when the sender "signals",
> all callbacks are dealt with by those who have registered.
Where did you find the information on the callbacks? I can't find
anything in the standard
>From page 448 of UML Superstructure:
"A signal is a specification of send request instances communicated
between objects"
from page 449:
"A signal triggers a reaction in the receiver in an asynchronous way
and without a reply. The sender of a signal will not block waiting
for a reply but continue execution immediately."
So I agree that UML::Signal matches Peters "Message" very well.
> An UML::event is something that *occurs*, but which is not
> (necessarily?) related to inter-object communication. Hence, in
> statemachines, one can trigger a transition based on the occurence of
> a time-event (e.g. 10 secs have elapsed), or a call-event, ...
Agreed.
from page 440:
"An event is the specification of some occurrence that may potentially
trigger effects by an object."
This is an abstract metaclass, which is specialized by TimeEvent,
SignalEvent, ...
It is somewhat misleading that an UML::Event specifies a *type* and
not an occurence.
> As I said in my original post, it ain't 100% clear to me either :(.
> But I think that in orocos currently everything is expressed as a
> signal (hence a mapping between RTT::Event and UML::Signal)
Yes, I agree. A RTT::Event maps to a UML::Signal and upon reception
leads to the occurance of a UML::SignalEvent which can be used in
statemachines.
Best regards
Markus
The road to RTT 2.0: towards a sustainable distributed component
On Tue, Feb 17, 2009 at 9:46 AM, Markus Klotzbücher
<markus [dot] klotzbuecher [..] ...> wrote:
On Mon, Feb 16, 2009 at 06:21:50PM +0100, Klaas Gadeyne wrote:
>> On Mon, Feb 16, 2009 at 5:53 PM, Sylvain Joyeux <sylvain [dot] joyeux [..] ...> wrote:
>> >> Can you rephrase/elaborate that question? I don't understand what you
>> >> mean with "the need *between*" in the sentence above.
>> > Sorry, missing some words.
>> >
>> > What I meant, is "the need to have a distinction between an event and a
>> > signal". In other words: what are *exactly* events and signals in UML terms ?
>>
>> [Disclaimer: I should _really_ check what I say below
>> Also see my original reply at <http://www.orocos.org/node/1068#comment-3103> ]
>>
>> If I have it right, an UML::Signal is an *inter-object communication
>> primitive* (a particular kind of message), a callback mechanism. One
>> object registers a callback (handler), and when the sender "signals",
>> all callbacks are dealt with by those who have registered.
>
> Where did you find the information on the callbacks? I can't find
> anything in the standard
Nowhere, it was my own way of expressing the RTT::Event principle :-)
Klaas
The road to RTT 2.0: towards a sustainable distributed component
> If I have it right, an UML::Signal is an *inter-object communication
> primitive* (a particular kind of message), a callback mechanism. One
> object registers a callback (handler), and when the sender "signals",
> all callbacks are dealt with by those who have registered.
>
> An UML::event is something that *occurs*, but which is not
> (necessarily?) related to inter-object communication. Hence, in
> statemachines, one can trigger a transition based on the occurence of
> a time-event (e.g. 10 secs have elapsed), or a call-event, ...
Okay ... So is not it that signals are just events that are externally
available while events are internal ?
I really don't see the point of having two objects here ...
The road to RTT 2.0: towards a sustainable distributed component
On Mon, Feb 16, 2009 at 6:34 PM, Sylvain Joyeux <sylvain [dot] joyeux [..] ...> wrote:
>> If I have it right, an UML::Signal is an *inter-object communication
>> primitive* (a particular kind of message), a callback mechanism. One
>> object registers a callback (handler), and when the sender "signals",
>> all callbacks are dealt with by those who have registered.
>>
>> An UML::event is something that *occurs*, but which is not
>> (necessarily?) related to inter-object communication. Hence, in
>> statemachines, one can trigger a transition based on the occurence of
>> a time-event (e.g. 10 secs have elapsed), or a call-event, ...
>
> Okay ... So is not it that signals are just events that are externally
> available while events are internal ?
>
> I really don't see the point of having two objects here ...
It's been too long since I worked with OSD (Markus, help! :-), but
AFAIR, orocos state machines expressed as OSD are now by their nature
asynchronous only (thereby avoiding synchronisation issues). Now
suppose we want to get rid of this "limitation", and e.g. also allow
transitions based on RTT::Methods (call based). AFAIR, this is
currently not possible (unless using "asynchronous workarounds").
If OSD transitions would be based on UML::events, this should become
possible (by the notion of a CallEvent, which occurs if someone
"calls" the method, and hence something which has _nothing_ to do with
UML::Signals ~ current RTT::Events.)
Note: I ain't saying it would be easy, since you would have to deal
with the synchronisation issues etc., but if you only have signals,
you remain with the asynchronous limitation I guess.
Klaas
The road to RTT 2.0: towards a sustainable distributed component
On Mon, Feb 16, 2009 at 07:04:17PM +0100, Klaas Gadeyne wrote:
> On Mon, Feb 16, 2009 at 6:34 PM, Sylvain Joyeux <sylvain [dot] joyeux [..] ...> wrote:
> >> If I have it right, an UML::Signal is an *inter-object communication
> >> primitive* (a particular kind of message), a callback mechanism. One
> >> object registers a callback (handler), and when the sender "signals",
> >> all callbacks are dealt with by those who have registered.
> >>
> >> An UML::event is something that *occurs*, but which is not
> >> (necessarily?) related to inter-object communication. Hence, in
> >> statemachines, one can trigger a transition based on the occurence of
> >> a time-event (e.g. 10 secs have elapsed), or a call-event, ...
> >
> > Okay ... So is not it that signals are just events that are externally
> > available while events are internal ?
> >
> > I really don't see the point of having two objects here ...
>
> It's been too long since I worked with OSD (Markus, help! :-), but
> AFAIR, orocos state machines expressed as OSD are now by their nature
> asynchronous only (thereby avoiding synchronisation issues). Now
You mean asynchronous with respect to the execution of the containing
TaskContext? It was not clear to me that the avoidance of
synchronization issues was a design goal.
> suppose we want to get rid of this "limitation", and e.g. also allow
> transitions based on RTT::Methods (call based). AFAIR, this is
> currently not possible (unless using "asynchronous workarounds").
> If OSD transitions would be based on UML::events, this should become
> possible (by the notion of a CallEvent, which occurs if someone
> "calls" the method, and hence something which has _nothing_ to do with
> UML::Signals ~ current RTT::Events.)
Yes. I think this would work fine, because in the case of a CallEvent
a method of our TaskContext is being executed anyway, so the
state machine could be progressed at the same time (before or after
the method call?)
It gets trickier for TimeEvents and non-periodic TaskContexts or
TaskContexts without activities at all. In whose thread of execution
is the state machine advanced on occurence of a TimeEvent?
> Note: I ain't saying it would be easy, since you would have to deal
> with the synchronisation issues etc., but if you only have signals,
> you remain with the asynchronous limitation I guess.
I agree it would be definitely desirable to be able to trigger state
machine transitions on a broader set of event occurences such as
defined by UML::Event.
Best regards
Markus
The road to RTT 2.0: towards a sustainable distributed component
On Tue, Feb 17, 2009 at 10:04 AM, Markus Klotzbücher
<markus [dot] klotzbuecher [..] ...> wrote:
> On Mon, Feb 16, 2009 at 07:04:17PM +0100, Klaas Gadeyne wrote:
>> It's been too long since I worked with OSD (Markus, help! :-), but
>> AFAIR, orocos state machines expressed as OSD are now by their nature
>> asynchronous only (thereby avoiding synchronisation issues). Now
>
> You mean asynchronous with respect to the execution of the containing
> TaskContext? It was not clear to me that the avoidance of
> synchronization issues was a design goal.
Sorry, should have expressed myself clearer. With asynchronous, I
meant that they are evaluated only in the TaskContext's own activity
(and hence asynchronous if some other component raises an event). I
don't think this was a "design goal". But I didn't design it, so...
:-)
[...]
> It gets trickier for TimeEvents and non-periodic TaskContexts or
> TaskContexts without activities at all. In whose thread of execution
> is the state machine advanced on occurence of a TimeEvent?
Exactly, much trickier, but you have to do *something* to earn your PhD :-)
best regards,
Klaas
The road to RTT 2.0: towards a sustainable distributed component
On Tuesday 10 February 2009 08:49:17 Klaas Gadeyne wrote:
>
> As a side note, I somehow fail to see the relation between the FOSDEM
> conf and the rest of your email? We have seen guys with arm boards
> with and without ethernet ports before right?
The spark is different than the fire.
> AutoSAR on FOSDEM? Most
> of the things below were (to my surprise) not really knew to me
> (unless I misunderstood them :-) and I felt like we discussed them
> before. Ofcourse, this makes it harder for me to disagree :-)
> This said...
My memory might be failing here. Can you point out a ML post about this topic?
>
> Enter UML spec:
> In UML, the "UML::Message" is the base primitive for interObject
> (think Sequence Diagram) communication. Apart from some exotic
> constructions, there are 3 "kinds" of Messages (see e.g. Fig 14.5 of
> the latest spec),
> - synchCall (mapping to what you call Method above)
> - asynchCall (mapping to what you call RTT::Message above)
> - asynchSignal (mapping "plus minus" to RTT::Event above)
>
> So:
> 1/ Since we are moving in a direction compliant with the spec, I like
> the above suggestions
I wonder how far we need to go with modelling controller applications using a
tool to model software. There are clearly similarities, but why did you cite
UML and not SysML for example ? Or any other domain specific language ?
I wanted to stress the difference between synchCall and asynchCall (how does
UML specify to implement this btw?) by giving them 'clearly' different names:
A message is sent, a method is called.
>
> 2/ However, it might be a good time to revise the naming? Was there
> anyone who intuitively understood the fact that a RTT::Method was
> something synchronous and the (now level 2) RTT::Command something
> asynchronous?
> Personnally, I think the UML names are more speaking-for-themselves
> than your suggestions of Message/Method (although that does not take
> into account our legacy users though)
I personally have no problem with deviating from a spec which most/all of our
users never have read and to which we claim no compliance. I don't want to
cause confusion neither of course. But that's something different.
>
> 3/ [I saved the most complex for last :]
> RTT::Events. I would describe this in a metaphor (heck I didn't even
> bother to check whether this is a correct englisch word) as a
> component (UML::Object) raising a "flag", so that others looking at
> the component can react appropriately (i.e. a primitive for _inter_
> object communication).
I would agree, although I would call it a primitive for decoupling inter-
object communication. The inter-object communication itself is done by
Message/Method.
>
> Entering Stephen commments about "Hey, I find them (rtt::events) very
> useful, especially in my FSMs". Exactly. However, FSM describes the
> behaviour of a certain component and not the inter-component
> communication. In UML, one has
> - Events (something happened):
== RTT::Event
> - Signals (interobject _communication_)
== RTT::Method/Message
>
> There is clearly a relationship between this (UML::SignalEvent), but
> In orocos 1.x (and in your current proposal), I have the gut feeling
> these 2 things are somehow mixed up (I dont' know whether this is
> necessarily a bad thing, or a way of implementing/interpreting the UML
> spec.)
> E.g. when we need a TimeEvent in our orocos FSM, we rely on a 3rd
> component that delivers us a UML::Signal.
That's because time events are no native part of RTT state machine scripts, we
modeled them as any other event.
>
> Maybe we should think wether the Event/Signal separation could also be
> useful in RTT 2.0
I agree completely. My proposal wanted to achieve this.
Peter
The road to RTT 2.0: towards a sustainable distributed component
On Wed, Feb 11, 2009 at 9:47 AM, Peter Soetens <peter [dot] soetens [..] ...> wrote:
> On Tuesday 10 February 2009 08:49:17 Klaas Gadeyne wrote:
>>
>> As a side note, I somehow fail to see the relation between the FOSDEM
>> conf and the rest of your email? We have seen guys with arm boards
>> with and without ethernet ports before right?
>
> The spark is different than the fire.
>
>> AutoSAR on FOSDEM? Most
>> of the things below were (to my surprise) not really knew to me
>> (unless I misunderstood them :-) and I felt like we discussed them
>> before. Ofcourse, this makes it harder for me to disagree :-)
>> This said...
>
> My memory might be failing here. Can you point out a ML post about this topic?
I didn't write "_email_ discussion", did I?
>> Enter UML spec:
>> In UML, the "UML::Message" is the base primitive for interObject
>> (think Sequence Diagram) communication. Apart from some exotic
>> constructions, there are 3 "kinds" of Messages (see e.g. Fig 14.5 of
>> the latest spec),
>> - synchCall (mapping to what you call Method above)
>> - asynchCall (mapping to what you call RTT::Message above)
>> - asynchSignal (mapping "plus minus" to RTT::Event above)
>>
>> So:
>> 1/ Since we are moving in a direction compliant with the spec, I like
>> the above suggestions
>
> I wonder how far we need to go with modelling controller applications using a
> tool to model software. There are clearly similarities, but why did you cite
> UML and not SysML for example ?
Because this part is exactly the same in both UML and SysML.
> Or any other domain specific language ?
Because there is no need to IMO. We're talking here about a software
feature, the difference between synchronous and asynchronous method
calls.
> I wanted to stress the difference between synchCall and asynchCall (how does
> UML specify to implement this btw?) by giving them 'clearly' different names:
> A message is sent, a method is called.
How could it be more explicit than by leaving synch and asynch _in_ their name?
>> 2/ However, it might be a good time to revise the naming? Was there
>> anyone who intuitively understood the fact that a RTT::Method was
>> something synchronous and the (now level 2) RTT::Command something
>> asynchronous?
>> Personnally, I think the UML names are more speaking-for-themselves
>> than your suggestions of Message/Method (although that does not take
>> into account our legacy users though)
>
> I personally have no problem with deviating from a spec which most/all of our
> users never have read and to which we claim no compliance. I don't want to
> cause confusion neither of course. But that's something different.
>
>>
>> 3/ [I saved the most complex for last :]
>> RTT::Events. I would describe this in a metaphor (heck I didn't even
>> bother to check whether this is a correct englisch word) as a
>> component (UML::Object) raising a "flag", so that others looking at
>> the component can react appropriately (i.e. a primitive for _inter_
>> object communication).
>
> I would agree, although I would call it a primitive for decoupling inter-
> object communication. The inter-object communication itself is done by
> Message/Method.
>
>>
>> Entering Stephen commments about "Hey, I find them (rtt::events) very
>> useful, especially in my FSMs". Exactly. However, FSM describes the
>> behaviour of a certain component and not the inter-component
>> communication. In UML, one has
>> - Events (something happened):
>
> == RTT::Event
I don't know.
>> - Signals (interobject _communication_)
>
> == RTT::Method/Message
No, since a UML Signal has no destination!
>>
>> There is clearly a relationship between this (UML::SignalEvent), but
>> In orocos 1.x (and in your current proposal), I have the gut feeling
>> these 2 things are somehow mixed up (I dont' know whether this is
>> necessarily a bad thing, or a way of implementing/interpreting the UML
>> spec.)
>> E.g. when we need a TimeEvent in our orocos FSM, we rely on a 3rd
>> component that delivers us a UML::Signal.
>
> That's because time events are no native part of RTT state machine scripts, we
> modeled them as any other event.
>
>>
>> Maybe we should think wether the Event/Signal separation could also be
>> useful in RTT 2.0
>
> I agree completely. My proposal wanted to achieve this.
I have the feeling we don't agree completely (see the above naming
confusing), but I hope I might be wrong :-)
Klaas
The road to RTT 2.0: towards a sustainable distributed component
On Wednesday 11 February 2009 12:10:17 Klaas Gadeyne wrote:
> On Wed, Feb 11, 2009 at 9:47 AM, Peter Soetens <peter [dot] soetens [..] ...> > >
My memory might be failing here. Can you point out a ML post about this
> > topic?
>
> I didn't write "_email_ discussion", did I?
If you can't find it with Google, it doesn't exist :-)
> >
> > I wonder how far we need to go with modelling controller applications
> > using a tool to model software. There are clearly similarities, but why
> > did you cite UML and not SysML for example ?
>
> Because this part is exactly the same in both UML and SysML.
>
> > Or any other domain specific language ?
>
> Because there is no need to IMO. We're talking here about a software
> feature, the difference between synchronous and asynchronous method
> calls.
Does UML make the distinction between 'method A' can only be called
synchronously and 'method B' only asynchronously ? Because a method is
supposed to be called in one case only, or its implementation has to change.
>
> > I wanted to stress the difference between synchCall and asynchCall (how
> > does UML specify to implement this btw?) by giving them 'clearly'
> > different names: A message is sent, a method is called.
>
> How could it be more explicit than by leaving synch and asynch _in_ their
> name?
Then propose a name.
> >
> >> Entering Stephen commments about "Hey, I find them (rtt::events) very
> >> useful, especially in my FSMs". Exactly. However, FSM describes the
> >> behaviour of a certain component and not the inter-component
> >> communication. In UML, one has
> >> - Events (something happened):
> >
> > == RTT::Event
>
> I don't know.
>
> >> - Signals (interobject _communication_)
> >
> > == RTT::Method/Message
>
> No, since a UML Signal has no destination!
This is absurd. A communication without a destination ?
I get the impression that UML models the low level, abstract primitives
required to setup a software system, ie Event - Signal - Operation. I'm not
ready to burden the Orocos user with the same complexity. I'm ready to listen
to the first Orocos user who claims to 'need' these primitives. I was aiming at
reducing complexity and primitives, not adding. I was aiming at solving the
confusion that exists around commands, the ignorance around events, and the
fragmentation of our component interface. UML was not in the picture. It
doesn't solve our problems.
Peter
The road to RTT 2.0: towards a sustainable distributed component
On Wed, Feb 11, 2009 at 02:37:37PM +0100, Peter Soetens wrote:
> Does UML make the distinction between 'method A' can only be called
> synchronously and 'method B' only asynchronously ? Because a method is
> supposed to be called in one case only, or its implementation has to change.
It does, IIRC this depends on if a object is active, e.g. an instance
of an active class (illustrated with a thick border) or not.
> I get the impression that UML models the low level, abstract primitives
> required to setup a software system, ie Event - Signal - Operation. I'm not
> ready to burden the Orocos user with the same complexity. I'm ready to listen
And you shouldn't! These are primitives after all, the building blocks
which get assembled in different ways to form higher level
abstractions - which then are immediately useful to the user! For
example a command has no corresponding UML entity, but why should it,
it can easily be constructed by a few RTT::Messages or asyncCalls :-)
> to the first Orocos user who claims to 'need' these primitives. I was aiming at
> reducing complexity and primitives, not adding. I was aiming at solving the
> confusion that exists around commands, the ignorance around events, and the
> fragmentation of our component interface. UML was not in the picture. It
> doesn't solve our problems.
I think a lot of confusion arises because of UML usage for the
implementation domain, which is the OO C++ language *and* the solution
domain which is the OROCOS component model! Of course the latter is
C++ too, but this is purely for technical reasons! These are really
different platforms. And we shouldn't expect to find appropriate UML
terms for all higher level abstractions that we construct. But if we
do as with RTT::Method for instance it might be helpful to point out
that this lower-level object oriented concept has been introduced in
our solution domain because it proved to be useful.
Best regards
Markus
The road to RTT 2.0: towards a sustainable distributed component
On Wed, 11 Feb 2009, Peter Soetens wrote:
[...]
> I get the impression that UML models the low level, abstract primitives
> required to setup a software system, ie Event - Signal - Operation. I'm not
> ready to burden the Orocos user with the same complexity. I'm ready to listen
> to the first Orocos user who claims to 'need' these primitives. I was aiming at
> reducing complexity and primitives, not adding. I was aiming at solving the
> confusion that exists around commands, the ignorance around events, and the
> fragmentation of our component interface. UML was not in the picture. It
> doesn't solve our problems.
This sounds a bit, euh.., arrogant?:-) UML has not been developed by a set
of unworldly computer scientists without a grasp of realworld use cases...
Of course, UML doesn't _solve_ our problems, but it can be a very important
help! In the sense that any concept that Orocos uses and that _can_ be
borrowed from UML brings with it a large gain in documentation clarity. If
we ignore UML, we will have to explain over and over again in what respects
the Orocos primitives are _different_ from the UML ones.
For clarity: I _know_ that UML can currently not solve _all_ the Orocos
problems...
Herman
The road to RTT 2.0: towards a sustainable distributed component
On Wed, Feb 11, 2009 at 2:37 PM, Peter Soetens <peter [dot] soetens [..] ...> wrote:
> On Wednesday 11 February 2009 12:10:17 Klaas Gadeyne wrote:
>> On Wed, Feb 11, 2009 at 9:47 AM, Peter Soetens <peter [dot] soetens [..] ...> > >
> My memory might be failing here. Can you point out a ML post about this
>> > topic?
>>
>> I didn't write "_email_ discussion", did I?
>
> If you can't find it with Google, it doesn't exist :-)
>
>> >
>> > I wonder how far we need to go with modelling controller applications
>> > using a tool to model software. There are clearly similarities, but why
>> > did you cite UML and not SysML for example ?
>>
>> Because this part is exactly the same in both UML and SysML.
>>
>> > Or any other domain specific language ?
>>
>> Because there is no need to IMO. We're talking here about a software
>> feature, the difference between synchronous and asynchronous method
>> calls.
>
> Does UML make the distinction between 'method A' can only be called
> synchronously and 'method B' only asynchronously ? Because a method is
> supposed to be called in one case only, or its implementation has to change.
Yes
>>
>> > I wanted to stress the difference between synchCall and asynchCall (how
>> > does UML specify to implement this btw?) by giving them 'clearly'
>> > different names: A message is sent, a method is called.
>>
>> How could it be more explicit than by leaving synch and asynch _in_ their
>> name?
>
> Then propose a name.
Hey, that's what I did in the first place. synchCall and asynchCall.
>> >> Entering Stephen commments about "Hey, I find them (rtt::events) very
>> >> useful, especially in my FSMs". Exactly. However, FSM describes the
>> >> behaviour of a certain component and not the inter-component
>> >> communication. In UML, one has
>> >> - Events (something happened):
>> >
>> > == RTT::Event
>>
>> I don't know.
>>
>> >> - Signals (interobject _communication_)
>> >
>> > == RTT::Method/Message
>>
>> No, since a UML Signal has no destination!
>
> This is absurd. A communication without a destination ?
Just as an Event to which no one is connected.
> I get the impression that UML models the low level, abstract primitives
> required to setup a software system, ie Event - Signal - Operation. I'm not
> ready to burden the Orocos user with the same complexity. I'm ready to listen
> to the first Orocos user who claims to 'need' these primitives. I was aiming at
> reducing complexity and primitives, not adding. I was aiming at solving the
> confusion that exists around commands, the ignorance around events, and the
> fragmentation of our component interface. UML was not in the picture. It
> doesn't solve our problems.
I get the impression you're stuck in a "UML is a bad, I don't want
anything from it"
Unfortunately, the world is not white nor black, but something in
between. So is UML...
Klaas
The road to RTT 2.0: towards a sustainable distributed component
> >> >> - Events (something happened):
> >> >
> >> > == RTT::Event
> >>
> >> I don't know.
> >>
> >> >> - Signals (interobject _communication_)
> >> >
> >> > == RTT::Method/Message
> >>
> >> No, since a UML Signal has no destination!
> >
> > This is absurd. A communication without a destination ?
>
> Just as an Event to which no one is connected.
I don't see the point of signals (w.r.t. events). Could you explain a bit more
? I really don't see the need between events and signals (as they are in UML,
I'm not speaking of Orocos events here).
The road to RTT 2.0: towards a sustainable distributed component
On Mon, Feb 16, 2009 at 4:55 PM, Sylvain Joyeux <sylvain [dot] joyeux [..] ...> wrote:
>> >> >> - Events (something happened):
>> >> >
>> >> > == RTT::Event
>> >>
>> >> I don't know.
>> >>
>> >> >> - Signals (interobject _communication_)
>> >> >
>> >> > == RTT::Method/Message
>> >>
>> >> No, since a UML Signal has no destination!
>> >
>> > This is absurd. A communication without a destination ?
>>
>> Just as an Event to which no one is connected.
>
> I don't see the point of signals (w.r.t. events). Could you explain a bit more
> ? I really don't see the need between events and signals (as they are in UML,
> I'm not speaking of Orocos events here).
Can you rephrase/elaborate that question? I don't understand what you
mean with "the need *between*" in the sentence above.
Klaas
The road to RTT 2.0: towards a sustainable distributed component
On Wed, 11 Feb 2009, Klaas Gadeyne wrote:
[...]
>> == RTT::Method/Message
>
> No, since a UML Signal has no destination!
The property of having no destination is very useful in a _component based_
development and design context: it prevents people from including hard
dependencies on the destination identification, and hence to resort to a
"broker" to make connections. I think this is a good thing to have
(although it complicates things in the most simplistic contexts).
Herman
The road to RTT 2.0: towards a sustainable distributed component
On Wed, 11 Feb 2009, Peter Soetens wrote:
> On Tuesday 10 February 2009 08:49:17 Klaas Gadeyne wrote:
[...]
>> Enter UML spec:
>> In UML, the "UML::Message" is the base primitive for interObject
>> (think Sequence Diagram) communication. Apart from some exotic
>> constructions, there are 3 "kinds" of Messages (see e.g. Fig 14.5 of
>> the latest spec),
>> - synchCall (mapping to what you call Method above)
>> - asynchCall (mapping to what you call RTT::Message above)
>> - asynchSignal (mapping "plus minus" to RTT::Event above)
>>
>> So:
>> 1/ Since we are moving in a direction compliant with the spec, I like
>> the above suggestions
>
> I wonder how far we need to go with modelling controller applications using a
> tool to model software. There are clearly similarities, but why did you cite
> UML and not SysML for example ? Or any other domain specific language ?
These are relevant questions, for which we have to find an appropriate
trade-off for 2.0 or 3.0 :-) I do think that neglecting the standards is
not a clever policy, but blindly adopting is too restrictive since we know
already that they can currently not cover all our needs and use cases.
> I wanted to stress the difference between synchCall and asynchCall (how does
> UML specify to implement this btw?)
UML is (but definition?) not about _implementations_...
> by giving them 'clearly' different names:
> A message is sent, a method is called.
I am all in favour of optimal naming! But I am willing to sacrifice this
optimality if a widely adopted standard is defining terms already: we will
never be able to beat their documentation or their community penetration.
>> 2/ However, it might be a good time to revise the naming? Was there
>> anyone who intuitively understood the fact that a RTT::Method was
>> something synchronous and the (now level 2) RTT::Command something
>> asynchronous?
>> Personnally, I think the UML names are more speaking-for-themselves
>> than your suggestions of Message/Method (although that does not take
>> into account our legacy users though)
>
> I personally have no problem with deviating from a spec which most/all of our
> users never have read and to which we claim no compliance. I don't want to
> cause confusion neither of course. But that's something different.
I am, much more than you, concerned about _future_ users! And for them,
standards compliance _is_ important. I still believe we can serve both (or
even more) communities by clever configurability of the framework: a
"standards compliant" building should be possible, even if that would mean
that one cannot profit from the nicest features or the most efficient
implementations of Orocos.
>> 3/ [I saved the most complex for last :]
>> RTT::Events. I would describe this in a metaphor (heck I didn't even
>> bother to check whether this is a correct englisch word) as a
>> component (UML::Object) raising a "flag", so that others looking at
>> the component can react appropriately (i.e. a primitive for _inter_
>> object communication).
>
> I would agree, although I would call it a primitive for decoupling inter-
> object communication. The inter-object communication itself is done by
> Message/Method.
Appropriate clarification, thanks!
>> Entering Stephen commments about "Hey, I find them (rtt::events) very
>> useful, especially in my FSMs". Exactly. However, FSM describes the
>> behaviour of a certain component and not the inter-component
>> communication. In UML, one has
>> - Events (something happened):
>
> == RTT::Event
>
>> - Signals (interobject _communication_)
>
> == RTT::Method/Message
What point do you want to make with these "equality assignment remarks"...?
That features match one-on-one between Orocos and UML? That would be
_great_, in my opinion... Taking into account that UML is only a
specification and not am implementation, and that UML has incomplete
semantics. So, let's focus on these latter things, and adopt the "UML
speak" where possible...
>> There is clearly a relationship between this (UML::SignalEvent), but
>> In orocos 1.x (and in your current proposal), I have the gut feeling
>> these 2 things are somehow mixed up (I dont' know whether this is
>> necessarily a bad thing, or a way of implementing/interpreting the UML
>> spec.)
>> E.g. when we need a TimeEvent in our orocos FSM, we rely on a 3rd
>> component that delivers us a UML::Signal.
>
> That's because time events are no native part of RTT state machine scripts, we
> modeled them as any other event.
Which is a "Good Thing" in my opinion, since it fits in your "Level 0,
Level 1, Level 2" abstraction levels, isn't it? In other words, this is
about how _reuse_ as much as possible from one _implementation_ to provide
as much as possible for several _specifications_...
>> Maybe we should think wether the Event/Signal separation could also be
>> useful in RTT 2.0
>
> I agree completely. My proposal wanted to achieve this.
I have the impression that overall agreement is huge, but that the most
appropriate description and wordings have not yet been sorted out... This
seems to be the ideal circumstances for a "version 0.2" of your proposal :-)
Herman
The road to RTT 2.0: towards a sustainable distributed component
On Tue, 10 Feb 2009, Klaas Gadeyne wrote:
> On Mon, Feb 9, 2009 at 12:26 AM, Peter Soetens <peter [dot] soetens [..] ...> wrote:
>> It's been FOSDEM last weekend and as is every year, this two-day conference
>> was a huge source of inspiration and reflection. Listening to what other people
>> do and need offers many opportunities for cooperation. And that's exactly what
>> this project is for. So naturally, I wondered how other projects could benefit
>> from the RTT.
>>
[...]
> Enter UML spec:
> In UML, the "UML::Message" is the base primitive for interObject
> (think Sequence Diagram) communication. Apart from some exotic
> constructions, there are 3 "kinds" of Messages (see e.g. Fig 14.5 of
> the latest spec),
> - synchCall (mapping to what you call Method above)
> - asynchCall (mapping to what you call RTT::Message above)
> - asynchSignal (mapping "plus minus" to RTT::Event above)
>
> So:
> 1/ Since we are moving in a direction compliant with the spec, I like
> the above suggestions
Me too!
> 2/ However, it might be a good time to revise the naming? Was there
> anyone who intuitively understood the fact that a RTT::Method was
> something synchronous and the (now level 2) RTT::Command something
> asynchronous?
> Personnally, I think the UML names are more speaking-for-themselves
> than your suggestions of Message/Method (although that does not take
> into account our legacy users though)
_If_ we change the naming, adopting the standardized UML names seems to be
the logical way to go.
> 3/ [I saved the most complex for last :]
> RTT::Events. I would describe this in a metaphor (heck I didn't even
> bother to check whether this is a correct englisch word) as a
> component (UML::Object) raising a "flag", so that others looking at
> the component can react appropriately (i.e. a primitive for _inter_
> object communication).
It is (should be) also a primitive for inter-_component_ communication...
> Entering Stephen commments about "Hey, I find them (rtt::events) very
> useful, especially in my FSMs". Exactly. However, FSM describes the
> behaviour of a certain component and not the inter-component
> communication. In UML, one has
> - Events (something happened):
> - Signals (interobject _communication_)
UML has the concept of "activity diagrams" and if I am not mistaken, those
cover our "component" aspects...
> There is clearly a relationship between this (UML::SignalEvent), but
> In orocos 1.x (and in your current proposal), I have the gut feeling
> these 2 things are somehow mixed up (I dont' know whether this is
> necessarily a bad thing, or a way of implementing/interpreting the UML
> spec.)
> E.g. when we need a TimeEvent in our orocos FSM, we rely on a 3rd
> component that delivers us a UML::Signal.
>
> Maybe we should think wether the Event/Signal separation could also be
> useful in RTT 2.0
>
> Klaas (going back to sleep now :-)
Thanks, you deserve it! :-)
One more thought that crossed my mind this night: I have (when I was as
young as most of you guys now) been working with message-based "middleware"
(although that name did not exist yet at that time) and I have learned one
important lesson: do _not_ give pure messages to "end users", since they
will introduce tremendous amount of coupling in their software, especially
via hidden ways such as hard coding the "names" of their communication
partners in their 'components' (another term that did not exist in those
days). It is worthwhile to discuss the "Broker vs. Brokerless" white paper
of ZeroMQ <http://www.zeromq.org/whitepapers:brokerless> in this
context....
Herman
The road to RTT 2.0: towards a sustainable distributed component
On Mon, 9 Feb 2009, Peter Soetens wrote:
Thanks Peter, for starting this very useful "roadmap" discussion! As ever,
I will react in a way that seems to suggest I do not agree with you, but,
in the end, as ever, it will turn out we do agree on almost everything :-)
[...]
> * RTT interoperates badly with other software, for example, any external
> process needs to go through a convoluted CORBA layer. There are also no tools
> that could ease the job (except the ctaskbrowser), for example some small
> shell commands that can query/change a component.
I think your are saying that "currently, RTT only has a working
implementation in CORBA that can be used to connect remotely", and _not_
"CORBA is a hard built-in dependency that will have to be used forever to
connect remotely"...?
> * RTT has remaining usability issues. Sylvain already identified the short
> commings of data/buffer ports and proposed a solution. But any user wrestling
> with the ''Should I use an Event (syn/asyn)-Method-Command-DataPort'?'
> question only got the answer: ''Well, we got Events(syn/asyn)-Methods-
> Commands and DataPorts !'. It's not coherent. There are other frameworks doing
> a better job.
Examples? Concrete analysis in what respects they are doing better?
But I agree with you that "Usability" should be high on the agenda, for the
coming years. Now that we have gained some mass of interested users outside
of the original development team, we should profit from that occasion, and
listen to use cases that are hard to do with Orocos now.
On the other hand, my experience is that most people have usability
problems because they do not see very well the difference between
synchronous and asynchronous component interactions, or with the
differences between object-oriented and component-based system design.
> We can do a far better job.
That's obvious :-)
> * RTT has issues with its current distribution implementation: programs
> can be constructed as such that they cause mem leaks at the remote side,
> Events never got into the CORBA interface (there is a reason for that),
> and our data ports over CORBA are equally weak as the C++ implementation.
Could these issues be sorted into more workable "levels"? I mean, some
issues come from baked-in design tradeoffs, others from incomplete
implementations... And the different "levels" require different reactions,
and can have different priorities. Thus could help us with the 2.0 Roadmap.
> * And then there are also the untaken opportunities to reduce RTT & component
> code size drastically and remove complex features.
Agreed. Or rather, to put them in more appropriate, decoupled modules, that
can be configured in or out.
> What does this all mean ? Start over ? Not at all. I'll defend that with
> small changes (of which some break backwards compatibility) we can solve
> all the above in one swift sweep. Unfortunately, it all starts with
> adding a new (but optionally optional) feature: rt_malloc, aka TLSF.
Do you suggest this is highest on the priority list? Could be, but why
exactly...?
> Yes
> you saw me write I would remove features, by adding a feature (well,
> actually two). I can explain.
> I've seen people using the RTT for inter-thread communication in two major
> ways: or implement a function as a Method, or as a Command. Where the command
> was the thread-safe way to change the state of a component. The adventurous
> used Events as well, but I can't say they're a huge success (we got like only
> one 'thank you' email in its whole existence...).
Events _are_ a huge success, but so obvious that people forget to thank you
for it! :-)
> But anyway, Commands are complex for newbies, Events (syn/asyn) aren't
> better.
I would phrase (and document!) it differently:
- these programming primitives are _essential_ for asynchronous components,
but overkill for synchronous ones.
- "newbies" should get a better introduction from us about the different
use cases for synchronous and asynchronous development.
- RTT should get support from a tookchain that provides "code generation"
in one way or another to solve the IPC interactions.
> So for all these people,
> here it comes: the RTT::Message object. Remember, Methods allow a peer
> component to _call_ a function foo(args) of the component interface. Messages
> will have the meaning of _sending_ another component a message to execute a
> function foo(args). Contrary to Methods, Messages are 'send and forget', they
> return void. The only guarantee you got is, that if the receiver was active,
> it processed it.
I have been advocating (asynchronous!) Message passing for a long time
already as _the_ basic IPC primitive, so I support their getting a more
prominent role in RTT, especially since all other programming primitives
can be implemented on top of them. Some good prior art exists, such as QNX
or <http://www.zeromq.org />. On the other hand, I would not like to see
message passing as the only primitive that RTT supports! Because Events,
Commands and Methods are very useful "policies" on top of asynchronous
message passing, and providing a waterthight implementation for them is
_very_ useful. Very useful...
> For now, forget that Commands exist.
Oops, that's one step too far! :-)
> We have two inter- component messaging primitives now: Messages and
> Methods.
I don't think that Methods in RTT have a message passing semantics now?!
They _are_ synchronous in their semantics. And their implementation
_should_ be synchronous for collocated, synchronous components (helping the
developer via the above-mentioned toolchain to generate automatically the
appropriate calling code and infrastructure!) while asynchronous message
passing is an appropriate way to implement synchronous(ly looking) Method
calls for distributed components.
> And each component declares: You can call these methods and send
> these messages.
> They are the 'Level 0' primitives of the RTT. Any transport should support
> these.
I _think_ we agree, but I am not sure that I fully understand your
sentences.
> Note that conveniently, the transport layer may implement messages with
> the same primitive as data ports. But we, users, don't care. We still have
> Data Ports to 'broadcast' our data streams and now we have Messages as well to
> send directly to component X.
This usage of Messages comes very close to the current "Event" semantics of
Orocos, with the slight addition that the Event can carry "data".
> Think about it. The RTT would be highly usable if each component only had
> data ports and a Message/Method interface. Ask the AUTOSAR people, it's
> very close to what they have (and can live with).
I agree, but like to see the "two levels" you mention _explicitly_ in the
RTT interface, but with appropriate documentation (the "working system
demoes" that Roderick is advocating, an idea that I thoroughly support).
> There's one side effect of the Message: we will need a real-time memory
> allocator to reserve a piece of memory for each message sent, and free it when
> the message is processed. Welcome TLSF.
Wait a bit, not so fast!!! As in every realtime system, it would be
acceptable practice to pre-allocate everything you think you need, and not
send any more messages if the pre-allocated memory is full. So, TLSF is
_one_ of the possible policies (which I support!), while the other is the
classical policy of "low-water - high-water" buffer allocation that has
proven to work in highly coupled embedded stuff (such as device drivers
with PCI support, and many others). Hence: okay for the introduction of
TLSF, but not as the only option.
> In case such a thing is not possible
> wanted by the user, Messages can fall back to using pre-allocated memory, but
> at the cost of reduced functionality (similar to what Commands can do today).
I do not like the "fall back" terminology! You seem to describe the
pre-allocated memory design as inferior, which it is not! When appropriate,
it is _superior_, and as such should be an integral part of RTT.
> Also, we'll have a MessageProcessor, which replaces and is a slimmed down
> version of the CommandProcessor today.
It is a _lower level_ infrastructure, that provides a _mechanism_ on top of
which Commands (and other _higher level_ _policies_) can be implemented.
> So where does this leave Events? Events are of the last primitives I explain
> in courses because they are so complex. They don't need to be. Today you need
> to attach a C/C++ function to an event and optionally specify an
> EventProcessor. Depending on some this-or-thats the function is executed in
> this-or-the-other thread. Let's forget about that. In essence, an Event is a
> local thing that others like to know about: Something happened 'here', who
> wants to know ? Events can be changed such that you can say: If event 'e'
> happens, then call this Method. And you can say: if event 'e' happens, send me
> this Message. You can subscribe as many callbacks as you want.
> Because of the lack of this mechanism, the current Event implementation has a
> huge foot print. There's a lot to win here.
I do not understand exactly in what respects the current "attachment of a
C/C++ function to an event" differs from what you now advocate as " If
event 'e' happens, then call this Method"?
> Do you want to allow others to raise the event ? Easy: add it to the Message
> or Method interface, saying: send me this Message and I'll raise the event, or
> call this Method and you'll raise it, respectively. But if someone can raise
> it, is your component's choice. That's what the event interface should look
> like. It's a Level 1. A transport should do no more than allowing to connect
> Methods and Messages (which it already supports, Level 1) to Events. No more.
> Even our CORBA layer could do that.
It might be useful to connect to some form of well-established
communication standard nomenclature to define what "Level 0", "Level 1",
etc would mean...
> The implementation of Event can benefit from a rt_malloc as well. Indirectly.
> Each raised Event which causes Messages to be sent out will use the Message's
> rt_malloc to store the event data by just sending the Message.
This would be _one_ of at least two possible implementations, the
pre-allocated memory buffers being the other one.
> In case you
> don't have/want an rt_malloc, you fall back to what events can roughly do
> today. But with lots of less code ( Goodbye RTT::ConnectionC, Goodbye
> RTT::EventProcessor ).
Is it really "goodbye!" or just "see you somewhere else, on a higher
Level!"?
> And now comes the climax: Sir Command. How does he fit in the picture? He'll
> remain in some form, but mainly as a 'Level 2' citizen.
This fits with my comments! (As I knew beforehand, since we have discussed
this matter several times in the past already :-)) But: what is "Level
2"...?
> He'll be composed of
> Methods, Messages and Events and will be dressed out to be no more than a
> wrapper, keeping related classes together or even not that. Replacing a
> Command with a Message hardly changes anything in the C++ side. For scripts,
> Commands were damn useful, but we will come up with something satisfactory.
> I'm sure.
I am not so sure. Commands have a specific semantics that _is_ useful in
many usecases. "Client-server" being one of them.
> How does all this interface shuffling allows us to get 'towards a
> sustainable distributed component model'? That's because we're seriously
> lowering the requirements on the transport layer:
Indeed! By finally letting RTT do it the way that good prior art (QNX, ...)
did it :-) I have no objection at all of following this prior art :-)
> * It only needs to implement the Level 0 primitives. How proxies and servers
> are built depends on the transport. You can do so manually (dlib like) or
> automatically (CORBA like)
> * It allows the transport to control memory better, share it between clients
> and clean it up at about any time.
> * The data flow changes Sylvain proposes strengthen our data flow model and I'm
> betting on it that it won't use CORBA as a transport. Who knows.
>
> And we are at the same time lowering the learning curve for new users:
> * You can easily explain the basic primitives: Properties=>XML,
> DataPorts=>process data, Methods/Messages=>client/server requests.
Oops! You are oversimplifying here!
> When
> they're familiar with these, they can start playing with Events (which build
> on top of Method/Messages and play a role in DataPorts as well). And finally,
> if they'll ever need, the Convoluted Command can encompass the most complex
> scenarios.
I think there is a conceptual flow in your thinking... Orocos/RTT should
attract "newbies" that _know_ why they come to Orocos, because also the
"complex, non-newbie" features are well supported. I do not believe in
educating users, from the (real) newbie level up...
> * You can more easily connect with other middleware or external programs.
> People with other middleware will see the opportunities for 1-to-1 mappings or
> even implement it as a transport in the RTT.
I agree!
> So this is my plan for RTT 2.0(09): not UML state machines, not generated
> code, not lua integration. They can wait for 3.0. Cleaning up the interface,
> unifying it, providing all users a smooth upgrade path, being similar to what
> other component frameworks offer. Setting a path out for growth. That's 2.0.
I agree!
> These major steps won't keep us from improving what we have, There's a lot
> we're keeping the same. I'll setup a RTT 2.0 branch with git. By next FOSDEM,
> I'd like to demonstrate it. In 45 minutes. With a small ARM board. Running one
> component. which I conveniently control from my command line, web interface or
> taskbrowser on my Laptop. Or any other laptop in the audience.
>
> It'll be fun.
It will! But please, by the next FOSDEM, the 3.0 features would be welcome
too, since there _are_ power users out there that want to use them, and
have them very well supported in tool chains! From the latter point of
view, the soon-to-be-started BRICS project should provide the manpower to
help Orocos achieving this toolchain support ambition...
Thanks for starting this discussion!
Herman
The road to RTT 2.0: towards a sustainable distributed component
On Monday 09 February 2009 11:07:04 Herman Bruyninckx wrote:
> On Mon, 9 Feb 2009, Peter Soetens wrote:
>
> Thanks Peter, for starting this very useful "roadmap" discussion! As ever,
> I will react in a way that seems to suggest I do not agree with you, but,
> in the end, as ever, it will turn out we do agree on almost everything :-)
I wasn't expecting anything else.
>
> [...]
>
> > * RTT interoperates badly with other software, for example, any external
> > process needs to go through a convoluted CORBA layer. There are also no
> > tools that could ease the job (except the ctaskbrowser), for example some
> > small shell commands that can query/change a component.
>
> I think your are saying that "currently, RTT only has a working
> implementation in CORBA that can be used to connect remotely", and _not_
> "CORBA is a hard built-in dependency that will have to be used forever to
> connect remotely"...?
Yes. It's fully pluggable (even after compilation), but on the other hand,
it's the only implementation we have.
>
> > * RTT has remaining usability issues. Sylvain already identified the
> > short commings of data/buffer ports and proposed a solution. But any user
> > wrestling with the ''Should I use an Event
> > (syn/asyn)-Method-Command-DataPort'?' question only got the answer:
> > ''Well, we got Events(syn/asyn)-Methods- Commands and DataPorts !'. It's
> > not coherent. There are other frameworks doing a better job.
>
> Examples? Concrete analysis in what respects they are doing better?
Simplicity is a big one. Fewer primitives which allow to build more complex
primitives.
>
> On the other hand, my experience is that most people have usability
> problems because they do not see very well the difference between
> synchronous and asynchronous component interactions, or with the
> differences between object-oriented and component-based system design.
That's why I want to 'drop' the terms syn/asyn. I would only talk in terms of
Methods and Messages (see below for more remarks).
>
> > * RTT has issues with its current distribution implementation: programs
> > can be constructed as such that they cause mem leaks at the remote side,
> > Events never got into the CORBA interface (there is a reason for that),
> > and our data ports over CORBA are equally weak as the C++ implementation.
>
> Could these issues be sorted into more workable "levels"? I mean, some
> issues come from baked-in design tradeoffs, others from incomplete
> implementations... And the different "levels" require different reactions,
> and can have different priorities. Thus could help us with the 2.0 Roadmap.
We'll do this on the Wiki.
>
> > What does this all mean ? Start over ? Not at all. I'll defend that with
> > small changes (of which some break backwards compatibility) we can solve
> > all the above in one swift sweep. Unfortunately, it all starts with
> > adding a new (but optionally optional) feature: rt_malloc, aka TLSF.
>
> Do you suggest this is highest on the priority list? Could be, but why
> exactly...?
We're having some awkward implementations, which impose illogical restrictions
on the user interface. The main thing we are missing because of no rt_malloc
is that we can't work message based. To put it otherways: today there's no
means of getting a piece of pre-allocated memory during communication. We only
have the not real-time malloc/free... which we can't use.
> > But anyway, Commands are complex for newbies, Events (syn/asyn) aren't
> > better.
For events, I was mainly speaking about the API, not about the functionality.
>
> I would phrase (and document!) it differently:
> - these programming primitives are _essential_ for asynchronous components,
> but overkill for synchronous ones.
> - "newbies" should get a better introduction from us about the different
> use cases for synchronous and asynchronous development.
I'd like to demonstrate it with the Message vs Method primitives
> - RTT should get support from a tookchain that provides "code generation"
> in one way or another to solve the IPC interactions.
>
> > So for all these people,
> > here it comes: the RTT::Message object. Remember, Methods allow a peer
> > component to _call_ a function foo(args) of the component interface.
> > Messages will have the meaning of _sending_ another component a message
> > to execute a function foo(args). Contrary to Methods, Messages are 'send
> > and forget', they return void. The only guarantee you got is, that if the
> > receiver was active, it processed it.
>
> I have been advocating (asynchronous!) Message passing for a long time
> already as _the_ basic IPC primitive, so I support their getting a more
> prominent role in RTT, especially since all other programming primitives
> can be implemented on top of them. Some good prior art exists, such as QNX
> or <http://www.zeromq.org />. On the other hand, I would not like to see
> message passing as the only primitive that RTT supports! Because Events,
> Commands and Methods are very useful "policies" on top of asynchronous
> message passing, and providing a waterthight implementation for them is
> _very_ useful. Very useful...
For middleware, Methods are implemented with a request-reply messages. In the
same memory space, they map to functions.
> > We have two inter- component messaging primitives now: Messages and
> > Methods.
>
> I don't think that Methods in RTT have a message passing semantics now?!
They do have at the lowest level of the CORBA implementation (TCP/IP). But for
plain in-process communications they map to functions.
> They _are_ synchronous in their semantics. And their implementation
> _should_ be synchronous for collocated, synchronous components (helping the
> developer via the above-mentioned toolchain to generate automatically the
> appropriate calling code and infrastructure!) while asynchronous message
> passing is an appropriate way to implement synchronous(ly looking) Method
> calls for distributed components.
Yes. We agree.
>
> > And each component declares: You can call these methods and send
> > these messages.
> > They are the 'Level 0' primitives of the RTT. Any transport should
> > support these.
>
> I _think_ we agree, but I am not sure that I fully understand your
> sentences.
Although the lowest level of the transport does everything message based, I
don't want to hide messages from the user API, nor do I want to force the user
to specify everything message based. The user actually doesn't care if there's
middleware or not involved. He just needs clear semantics: If I call function
A, x happens.
>
> > Note that conveniently, the transport layer may implement messages with
> > the same primitive as data ports. But we, users, don't care. We still
> > have Data Ports to 'broadcast' our data streams and now we have Messages
> > as well to send directly to component X.
>
> This usage of Messages comes very close to the current "Event" semantics of
> Orocos, with the slight addition that the Event can carry "data".
Messages cary data as well. When you connect a message to an event, the event
data is put into the message and sent to the receiver.
>
> > There's one side effect of the Message: we will need a real-time memory
> > allocator to reserve a piece of memory for each message sent, and free it
> > when the message is processed. Welcome TLSF.
>
> Wait a bit, not so fast!!! As in every realtime system, it would be
> acceptable practice to pre-allocate everything you think you need, and not
> send any more messages if the pre-allocated memory is full. So, TLSF is
> _one_ of the possible policies (which I support!), while the other is the
> classical policy of "low-water - high-water" buffer allocation that has
> proven to work in highly coupled embedded stuff (such as device drivers
> with PCI support, and many others). Hence: okay for the introduction of
> TLSF, but not as the only option.
Yes. And the other option is what I call the 'fall-back'.
>
> > In case such a thing is not possible
> > wanted by the user, Messages can fall back to using pre-allocated memory,
> > but at the cost of reduced functionality (similar to what Commands can do
> > today).
>
> I do not like the "fall back" terminology! You seem to describe the
> pre-allocated memory design as inferior, which it is not! When appropriate,
> it is _superior_, and as such should be an integral part of RTT.
I agree. TLSF is no more than a smart way of pre-allocating memory and then
redistributing it again.
>
> > Also, we'll have a MessageProcessor, which replaces and is a slimmed down
> > version of the CommandProcessor today.
>
> It is a _lower level_ infrastructure, that provides a _mechanism_ on top of
> which Commands (and other _higher level_ _policies_) can be implemented.
Just to be clear: although the lowest levels of the middleware (Level -1) use
a message based transport, At the user API level, there are RTT::Message
objects as well (which will probably map 1:1 to a middleware message, but this
is hidden).
>
> > So where does this leave Events? Events are of the last primitives I
> > explain in courses because they are so complex. They don't need to be.
> > Today you need to attach a C/C++ function to an event and optionally
> > specify an EventProcessor. Depending on some this-or-thats the function
> > is executed in this-or-the-other thread. Let's forget about that. In
> > essence, an Event is a local thing that others like to know about:
> > Something happened 'here', who wants to know ? Events can be changed such
> > that you can say: If event 'e' happens, then call this Method. And you
> > can say: if event 'e' happens, send me this Message. You can subscribe as
> > many callbacks as you want.
> > Because of the lack of this mechanism, the current Event implementation
> > has a huge foot print. There's a lot to win here.
>
> I do not understand exactly in what respects the current "attachment of a
> C/C++ function to an event" differs from what you now advocate as " If
> event 'e' happens, then call this Method"?
'M'ethod with capital M. RTT::Method is a C++ class. Today you can construct
that class, add it to the TaskContext API, but you can't attach it to an
Event. You need to go through another syntactical construct. It's really
avoiding duplicate code here. Same for 'RTT::Message'.
>
> > Do you want to allow others to raise the event ? Easy: add it to the
> > Message or Method interface, saying: send me this Message and I'll raise
> > the event, or call this Method and you'll raise it, respectively. But if
> > someone can raise it, is your component's choice. That's what the event
> > interface should look like. It's a Level 1. A transport should do no more
> > than allowing to connect Methods and Messages (which it already supports,
> > Level 1) to Events. No more. Even our CORBA layer could do that.
>
> It might be useful to connect to some form of well-established
> communication standard nomenclature to define what "Level 0", "Level 1",
> etc would mean...
A drawing :-)
>
> > In case you
> > don't have/want an rt_malloc, you fall back to what events can roughly do
> > today. But with lots of less code ( Goodbye RTT::ConnectionC, Goodbye
> > RTT::EventProcessor ).
>
> Is it really "goodbye!" or just "see you somewhere else, on a higher
> Level!"?
As it looks now, really goodbye. There is no such thing anymore as an
EventProcessor: the event broadcasts Messages, and the MessageProcessors
process them. We're really simplifying/unifying here.
>
> > And now comes the climax: Sir Command. How does he fit in the picture?
> > He'll remain in some form, but mainly as a 'Level 2' citizen.
>
> This fits with my comments! (As I knew beforehand, since we have discussed
> this matter several times in the past already :-)) But: what is "Level
> 2"...?
'Built on top of Method,Message and Event.'
L 2: RTT Complex primitives RTT::Command (Uses mix of L0,1)
L 1: RTT Broadcast primitive: RTT::Event (Uses L 0 to deliver data)
L 0: RTT Basics: RTT::Method and RTT::Message (implemented with L -1)
L -1: MIDDLEWARE - TRANSPORTS : message based TCP/IP, UDP, unix sockets,...
>
> > He'll be composed of
> > Methods, Messages and Events and will be dressed out to be no more than a
> > wrapper, keeping related classes together or even not that. Replacing a
> > Command with a Message hardly changes anything in the C++ side. For
> > scripts, Commands were damn useful, but we will come up with something
> > satisfactory. I'm sure.
>
> I am not so sure. Commands have a specific semantics that _is_ useful in
> many usecases. "Client-server" being one of them.
There might be small semantic changes (return values) and syntactic changes
(how to construct one). We'll have to make this exercise together.
Peter
The road to RTT 2.0: towards a sustainable distributed component
On Mon, 9 Feb 2009, Peter Soetens wrote:
[...]
>>> * RTT has remaining usability issues. Sylvain already identified the
>>> short commings of data/buffer ports and proposed a solution. But any user
>>> wrestling with the ''Should I use an Event
>>> (syn/asyn)-Method-Command-DataPort'?' question only got the answer:
>>> ''Well, we got Events(syn/asyn)-Methods- Commands and DataPorts !'. It's
>>> not coherent. There are other frameworks doing a better job.
>>
>> Examples? Concrete analysis in what respects they are doing better?
>
> Simplicity is a big one. Fewer primitives which allow to build more complex
> primitives.
>
Simplicity as you describe it here is a two-sided sword! Orocos has always
had different levels of "users" in mind (the framework builder, the
component builder, and the system builder). Each should have a "simple" set
of primitives to build more complex ones. But what is simple and complex is
different at each level! The Framework builders benefit most from the
suggested "2.0 redesign"; the component builders will not, since at that
level having only asynchronous message passing without more abstract
primitives is too complex! Or rather, you will immediately see reinvention
of the "Method", "Event" and "Command" primitives, but done poorly, and in
incompatible ways...
>> On the other hand, my experience is that most people have usability
>> problems because they do not see very well the difference between
>> synchronous and asynchronous component interactions, or with the
>> differences between object-oriented and component-based system design.
>
> That's why I want to 'drop' the terms syn/asyn. I would only talk in terms of
> Methods and Messages (see below for more remarks).
Wrong! Don't fall into the trap that hiding the complex things behind
easy/simple interfaces will automagically make an inherent complex activity
(= making high-quality, reusable, reportable, portable,...) components into
a simple one...
[...]
>>> What does this all mean ? Start over ? Not at all. I'll defend that with
>>> small changes (of which some break backwards compatibility) we can solve
>>> all the above in one swift sweep. Unfortunately, it all starts with
>>> adding a new (but optionally optional) feature: rt_malloc, aka TLSF.
>>
>> Do you suggest this is highest on the priority list? Could be, but why
>> exactly...?
>
> We're having some awkward implementations, which impose illogical restrictions
> on the user interface. The main thing we are missing because of no rt_malloc
> is that we can't work message based.
I don't agree! Runtime memory allocation is _not_ a necessary condition for
asynchronous message passing! There is sufficient prior art in this respect
(I indicated some of it in my previous mail, such as PCI).
[...]
> They do have at the lowest level of the CORBA implementation (TCP/IP). But for
> plain in-process communications they map to functions.
Then there is something wrong with the CORBA implementation...?
[...]
> Although the lowest level of the transport does everything message based,
> I don't want to hide messages from the user API, nor do I want to force
> the user to specify everything message based. The user actually doesn't
> care if there's middleware or not involved. He just needs clear
> semantics: If I call function A, x happens.
These seems to be two contradictory ambitions...
>>> Note that conveniently, the transport layer may implement messages with
>>> the same primitive as data ports. But we, users, don't care. We still
>>> have Data Ports to 'broadcast' our data streams and now we have Messages
>>> as well to send directly to component X.
>>
>> This usage of Messages comes very close to the current "Event" semantics of
>> Orocos, with the slight addition that the Event can carry "data".
>
> Messages cary data as well. When you connect a message to an event, the event
> data is put into the message and sent to the receiver.
Of course. You read my argument backwards: messages _always_ carry data, so
it is straightforward to let Events also carry data :-)
>
>>
>>> There's one side effect of the Message: we will need a real-time memory
>>> allocator to reserve a piece of memory for each message sent, and free it
>>> when the message is processed. Welcome TLSF.
>>
>> Wait a bit, not so fast!!! As in every realtime system, it would be
>> acceptable practice to pre-allocate everything you think you need, and not
>> send any more messages if the pre-allocated memory is full. So, TLSF is
>> _one_ of the possible policies (which I support!), while the other is the
>> classical policy of "low-water - high-water" buffer allocation that has
>> proven to work in highly coupled embedded stuff (such as device drivers
>> with PCI support, and many others). Hence: okay for the introduction of
>> TLSF, but not as the only option.
>
> Yes. And the other option is what I call the 'fall-back'.
No, they should get _equal_ status! With some explanations in the
documentation about the most appropriate use cases (and usage methods) for
both.
>>> In case such a thing is not possible
>>> wanted by the user, Messages can fall back to using pre-allocated memory,
>>> but at the cost of reduced functionality (similar to what Commands can do
>>> today).
>>
>> I do not like the "fall back" terminology! You seem to describe the
>> pre-allocated memory design as inferior, which it is not! When appropriate,
>> it is _superior_, and as such should be an integral part of RTT.
>
> I agree. TLSF is no more than a smart way of pre-allocating memory and then
> redistributing it again.
>>> Also, we'll have a MessageProcessor, which replaces and is a slimmed down
>>> version of the CommandProcessor today.
>>
>> It is a _lower level_ infrastructure, that provides a _mechanism_ on top of
>> which Commands (and other _higher level_ _policies_) can be implemented.
>
> Just to be clear: although the lowest levels of the middleware (Level -1) use
> a message based transport, At the user API level, there are RTT::Message
> objects as well (which will probably map 1:1 to a middleware message, but this
> is hidden).
Be careful about mixing two levels in the same API... It would be
worthwhile having multiple APIs, as in most software stacks.
>>> So where does this leave Events? Events are of the last primitives I
>>> explain in courses because they are so complex. They don't need to be.
>>> Today you need to attach a C/C++ function to an event and optionally
>>> specify an EventProcessor. Depending on some this-or-thats the function
>>> is executed in this-or-the-other thread. Let's forget about that. In
>>> essence, an Event is a local thing that others like to know about:
>>> Something happened 'here', who wants to know ? Events can be changed such
>>> that you can say: If event 'e' happens, then call this Method. And you
>>> can say: if event 'e' happens, send me this Message. You can subscribe as
>>> many callbacks as you want.
>>> Because of the lack of this mechanism, the current Event implementation
>>> has a huge foot print. There's a lot to win here.
>>
>> I do not understand exactly in what respects the current "attachment of a
>> C/C++ function to an event" differs from what you now advocate as " If
>> event 'e' happens, then call this Method"?
>
> 'M'ethod with capital M. RTT::Method is a C++ class. Today you can construct
> that class, add it to the TaskContext API, but you can't attach it to an
> Event. You need to go through another syntactical construct. It's really
> avoiding duplicate code here. Same for 'RTT::Message'.
Ok!
[...]
>>> In case you
>>> don't have/want an rt_malloc, you fall back to what events can roughly do
>>> today. But with lots of less code ( Goodbye RTT::ConnectionC, Goodbye
>>> RTT::EventProcessor ).
>>
>> Is it really "goodbye!" or just "see you somewhere else, on a higher
>> Level!"?
>
> As it looks now, really goodbye. There is no such thing anymore as an
> EventProcessor: the event broadcasts Messages, and the MessageProcessors
> process them. We're really simplifying/unifying here.
Ok! (I do not fully understand yet, but I think I got the idea. And I like
it.)
>>> And now comes the climax: Sir Command. How does he fit in the picture?
>>> He'll remain in some form, but mainly as a 'Level 2' citizen.
>>
>> This fits with my comments! (As I knew beforehand, since we have discussed
>> this matter several times in the past already :-)) But: what is "Level
>> 2"...?
>
> 'Built on top of Method,Message and Event.'
>
> L 2: RTT Complex primitives RTT::Command (Uses mix of L0,1)
> L 1: RTT Broadcast primitive: RTT::Event (Uses L 0 to deliver data)
> L 0: RTT Basics: RTT::Method and RTT::Message (implemented with L -1)
> L -1: MIDDLEWARE - TRANSPORTS : message based TCP/IP, UDP, unix sockets,...
Ok.
>>> He'll be composed of
>>> Methods, Messages and Events and will be dressed out to be no more than a
>>> wrapper, keeping related classes together or even not that. Replacing a
>>> Command with a Message hardly changes anything in the C++ side. For
>>> scripts, Commands were damn useful, but we will come up with something
>>> satisfactory. I'm sure.
>>
>> I am not so sure. Commands have a specific semantics that _is_ useful in
>> many usecases. "Client-server" being one of them.
>
> There might be small semantic changes (return values) and syntactic changes
> (how to construct one). We'll have to make this exercise together.
Ok.
Go for it!
Herman
The road to RTT 2.0: towards a sustainable distributed component
Hi Peter,
As discussed yesterday I mostly agree...
On Mon, Feb 09, 2009 at 12:26:25AM +0100, Peter Soetens wrote:
>
> It's been FOSDEM last weekend and as is every year, this two-day conference
> was a huge source of inspiration and reflection. Listening to what other people
> do and need offers many opportunities for cooperation. And that's exactly what
> this project is for. So naturally, I wondered how other projects could benefit
> from the RTT.
>
> There was this guy with a board with an ARM and an Ethernet port. Assuming
> that his system was so embedded that no more than one Orocos component needed
> to run on it, I wondered how 'junior' would then communicate with the
> 'server'. He couldn't use some shell script to remotely toggle a property or
> to log some data off a data port (we don't got any). He couldn't even implement
> the protocol the RTT uses (remember, its CORBA) or needed dragging in huge
> libraries. Finally I wondered how usable the RTT is in such a setup. Well that
> is if we got that far explaining the poor fella the intricacies of the RTT
> component interface.
>
> People knowing me know that after such a case, something better gets proposed
> next. But I'd like to point out the pain points I was touching here:
>
> * RTT interoperates badly with other software, for example, any external
> process needs to go through a convoluted CORBA layer. There are also no tools
> that could ease the job (except the ctaskbrowser), for example some small
> shell commands that can query/change a component.
>
> * RTT has remaining usability issues. Sylvain already identified the short
> commings of data/buffer ports and proposed a solution. But any user wrestling
> with the ''Should I use an Event (syn/asyn)-Method-Command-DataPort'?'
> question only got the answer: ''Well, we got Events(syn/asyn)-Methods-
> Commands and DataPorts !'. It's not coherent. There are other frameworks doing
> a better job. We can do a far better job.
>
> * RTT has issues with its current distribution implementation: programs can be
> constructed as such that they cause mem leaks at the remote side, Events never
> got into the CORBA interface (there is a reason for that), and our data ports
> over CORBA are equally weak as the C++ implementation.
>
> * And then there are also the untaken opportunities to reduce RTT & component
> code size drastically and remove complex features.
>
> What does this all mean ? Start over ? Not at all. I'll defend that with small
> changes (of which some break backwards compatibility) we can solve all the
> above in one swift sweep. Unfortunately, it all starts with adding a new (but
> optionally optional) feature: rt_malloc, aka TLSF. Yes you saw me write I
> would remove features, by adding a feature (well, actually two). I can
> explain.
>
> I've seen people using the RTT for inter-thread communication in two major
> ways: or implement a function as a Method, or as a Command. Where the command
> was the thread-safe way to change the state of a component. The adventurous
> used Events as well, but I can't say they're a huge success (we got like only
> one 'thank you' email in its whole existence...). But anyway, Commands are
> complex for newbies, Events (syn/asyn) aren't better. So for all these people,
> here it comes: the RTT::Message object. Remember, Methods allow a peer
> component to _call_ a function foo(args) of the component interface. Messages
> will have the meaning of _sending_ another component a message to execute a
> function foo(args). Contrary to Methods, Messages are 'send and forget', they
> return void. The only guarantee you got is, that if the receiver was active,
> it processed it. For now, forget that Commands exist. We have two inter-
> component messaging primitives now: Messages and Methods. And each component
> declares: You can call these methods and send these messages.
> They are the 'Level 0' primitives of the RTT. Any transport should support
> these. Note that conveniently, the transport layer may implement messages with
> the same primitive as data ports. But we, users, don't care. We still have
> Data Ports to 'broadcast' our data streams and now we have Messages as well to
> send directly to component X.
>
> Think about it. The RTT would be highly usable if each component only had data
> ports and a Message/Method interface. Ask the AUTOSAR people, it's very close
> to what they have (and can live with).
>
> There's one side effect of the Message: we will need a real-time memory
> allocator to reserve a piece of memory for each message sent, and free it when
> the message is processed. Welcome TLSF. In case such a thing is not possible
> wanted by the user, Messages can fall back to using pre-allocated memory, but
> at the cost of reduced functionality (similar to what Commands can do today).
> Also, we'll have a MessageProcessor, which replaces and is a slimmed down
> version of the CommandProcessor today.
I must admit I still have a slightly uneasy feeling about making
non-optional use of the TLSF, it's like giving up full determinism for
an application. But if it allows to simplify the implementation
substantially, well i guess it's worth it. On a side note, I attended
the MaRTe OS ( http://marte.unican.es/) presentation yesterday and
they use it too...
The key issue, just as it applies to GFP_ATOMIC in the Linux Kernel,
is that anyone performing real-time allocations *must* be capable of
dealing gracefully if the allocation fails.
> So where does this leave Events? Events are of the last primitives I explain
> in courses because they are so complex. They don't need to be. Today you need
> to attach a C/C++ function to an event and optionally specify an
> EventProcessor. Depending on some this-or-thats the function is executed in
> this-or-the-other thread. Let's forget about that. In essence, an Event is a
> local thing that others like to know about: Something happened 'here', who
> wants to know ? Events can be changed such that you can say: If event 'e'
> happens, then call this Method. And you can say: if event 'e' happens, send me
> this Message. You can subscribe as many callbacks as you want.
> Because of the lack of this mechanism, the current Event implementation has a
> huge foot print. There's a lot to win here.
>
> Do you want to allow others to raise the event ? Easy: add it to the Message
> or Method interface, saying: send me this Message and I'll raise the event, or
> call this Method and you'll raise it, respectively. But if someone can raise
> it, is your component's choice. That's what the event interface should look
> like. It's a Level 1. A transport should do no more than allowing to connect
> Methods and Messages (which it already supports, Level 1) to Events. No more.
> Even our CORBA layer could do that.
>
> The implementation of Event can benefit from a rt_malloc as well. Indirectly.
> Each raised Event which causes Messages to be sent out will use the Message's
> rt_malloc to store the event data by just sending the Message. In case you
> don't have/want an rt_malloc, you fall back to what events can roughly do
> today. But with lots of less code ( Goodbye RTT::ConnectionC, Goodbye
> RTT::EventProcessor ).
>
> And now comes the climax: Sir Command. How does he fit in the picture? He'll
> remain in some form, but mainly as a 'Level 2' citizen. He'll be composed of
> Methods, Messages and Events and will be dressed out to be no more than a
> wrapper, keeping related classes together or even not that. Replacing a
> Command with a Message hardly changes anything in the C++ side. For scripts,
> Commands were damn useful, but we will come up with something satisfactory.
> I'm sure.
As mentioned previously I think that having such a hierarchy,
e.g. building higher level abstractions from primitives of component
interaction is crucial. It guarantees orthogonality and facilitates
understanding.
> How does all this interface shuffling allows us to get 'towards a sustainable
> distributed component model'? That's because we're seriously lowering the
> requirements on the transport layer:
> * It only needs to implement the Level 0 primitives. How proxies and servers
> are built depends on the transport. You can do so manually (dlib like) or
> automatically (CORBA like)
> * It allows the transport to control memory better, share it between clients
> and clean it up at about any time.
> * The data flow changes Sylvain proposes strengthen our data flow model and I'm
> betting on it that it won't use CORBA as a transport. Who knows.
>
> And we are at the same time lowering the learning curve for new users:
> * You can easily explain the basic primitives: Properties=>XML,
> DataPorts=>process data, Methods/Messages=>client/server requests. When
> they're familiar with these, they can start playing with Events (which build
> on top of Method/Messages and play a role in DataPorts as well). And finally,
> if they'll ever need, the Convoluted Command can encompass the most complex
> scenarios.
> * You can more easily connect with other middleware or external programs.
> People with other middleware will see the opportunities for 1-to-1 mappings or
> even implement it as a transport in the RTT.
>
> So this is my plan for RTT 2.0(09): not UML state machines, not generated
> code, not lua integration. They can wait for 3.0. Cleaning up the interface,
Uh, maybe not that long, but I agree with the priorities, all these
items will benefit from clean and (as stable as possible) interfaces.
Best regards
Markus
The road to RTT 2.0: towards a sustainable distributed component
On Monday 09 February 2009 10:50:56 Markus Klotzbücher wrote:
> Hi Peter,
>
> As discussed yesterday I mostly agree...
Being my primary reflection board and pointing out the holes in my initial
ideas really helped getting things clear. I have to credit that.
>
> On Mon, Feb 09, 2009 at 12:26:25AM +0100, Peter Soetens wrote:
> > There's one side effect of the Message: we will need a real-time memory
> > allocator to reserve a piece of memory for each message sent, and free it
> > when the message is processed. Welcome TLSF. In case such a thing is not
> > possible wanted by the user, Messages can fall back to using
> > pre-allocated memory, but at the cost of reduced functionality (similar
> > to what Commands can do today). Also, we'll have a MessageProcessor,
> > which replaces and is a slimmed down version of the CommandProcessor
> > today.
>
> I must admit I still have a slightly uneasy feeling about making
> non-optional use of the TLSF, it's like giving up full determinism for
> an application. But if it allows to simplify the implementation
> substantially, well i guess it's worth it. On a side note, I attended
> the MaRTe OS ( http://marte.unican.es/) presentation yesterday and
> they use it too...
>
> The key issue, just as it applies to GFP_ATOMIC in the Linux Kernel,
> is that anyone performing real-time allocations *must* be capable of
> dealing gracefully if the allocation fails.
That's rule number one. At this point in discussion, how TLSF will be used is
an implementation detail. We could let it grow with each created 'Message'
object in the system. We have options.
Peter
The road to RTT 2.0: towards a sustainable distributed component
Holy crap, that's the longest email I've seen on this list ...
On Feb 8, 2009, at 18:26 , Peter Soetens wrote:
> It's been FOSDEM last weekend and as is every year, this two-day
> conference
> was a huge source of inspiration and reflection. Listening to what
> other people
> do and need offers many opportunities for cooperation. And that's
> exactly what
> this project is for. So naturally, I wondered how other projects
> could benefit
> from the RTT.
>
> There was this guy with a board with an ARM and an Ethernet port.
> Assuming
> that his system was so embedded that no more than one Orocos
> component needed
> to run on it, I wondered how 'junior' would then communicate with the
> 'server'. He couldn't use some shell script to remotely toggle a
> property or
> to log some data off a data port (we don't got any). He couldn't
> even implement
> the protocol the RTT uses (remember, its CORBA) or needed dragging
> in huge
> libraries. Finally I wondered how usable the RTT is in such a setup.
> Well that
> is if we got that far explaining the poor fella the intricacies of
> the RTT
> component interface.
>
> People knowing me know that after such a case, something better gets
> proposed
> next. But I'd like to point out the pain points I was touching here:
>
> * RTT interoperates badly with other software, for example, any
> external
> process needs to go through a convoluted CORBA layer. There are also
> no tools
> that could ease the job (except the ctaskbrowser), for example some
> small
> shell commands that can query/change a component.
I love the Corba integration! It's butt easy ... problem is, we
provide no application level examples of how to use.
How much simpler can a GUI be than
find deployer ** close to a one-liner **
find peer ** ditto **
local_method = peer.getMethd()
...
local_method(a,b,c)
How much more do you want? The code in the GUI looks exactly like code
in local components. Sure, the semantics of timing, latency, etc, are
different, but it's going across IP one way or the other. Deal with it.
Now, writing new CORBA plugins is like being in hell or purgatory ...
that needs serious work ... my sanity will never be the same after
doing 2 of those ... and I've more to come ... :-(
> * RTT has remaining usability issues. Sylvain already identified the
> short
> commings of data/buffer ports and proposed a solution. But any user
> wrestling
> with the ''Should I use an Event (syn/asyn)-Method-Command-DataPort'?'
> question only got the answer: ''Well, we got Events(syn/asyn)-
> Methods-
> Commands and DataPorts !'. It's not coherent. There are other
> frameworks doing
> a better job. We can do a far better job.
There are a lot of pieces, agreed. It takes a long time to come up the
learning curve ... though each piece does have it's place.
> * RTT has issues with its current distribution implementation:
> programs can be
> constructed as such that they cause mem leaks at the remote side,
> Events never
> got into the CORBA interface (there is a reason for that), and our
> data ports
> over CORBA are equally weak as the C++ implementation.
How are data ports weak?
> * And then there are also the untaken opportunities to reduce RTT &
> component
> code size drastically and remove complex features.
>
> What does this all mean ? Start over ? Not at all. I'll defend that
> with small
You start over, and I sail all the way to Brussels and shoot your
sorry behind!
> changes (of which some break backwards compatibility) we can solve
> all the
> above in one swift sweep. Unfortunately, it all starts with adding a
> new (but
> optionally optional) feature: rt_malloc, aka TLSF. Yes you saw me
> write I
> would remove features, by adding a feature (well, actually two). I can
> explain.
TLSF would help greatly in some situations. Would also open the door
for abuse, but people have to learn somewhere.
> I've seen people using the RTT for inter-thread communication in two
> major
> ways: or implement a function as a Method, or as a Command. Where
> the command
> was the thread-safe way to change the state of a component. The
> adventurous
> used Events as well, but I can't say they're a huge success (we got
> like only
> one 'thank you' email in its whole existence...). But anyway,
> Commands are
Events are fantastic when combined with state machines!! Thank you,
thank you, thank you ... there, now you have 4 thank yous for
Events! ;-) I *regularly* use events to inter-connect state
machines. It makes this trivial.
> complex for newbies, Events (syn/asyn) aren't better. So for all
> these people,
> here it comes: the RTT::Message object. Remember, Methods allow a peer
> component to _call_ a function foo(args) of the component interface.
> Messages
> will have the meaning of _sending_ another component a message to
> execute a
> function foo(args). Contrary to Methods, Messages are 'send and
> forget', they
> return void. The only guarantee you got is, that if the receiver was
> active,
> it processed it. For now, forget that Commands exist. We have two
> inter-
> component messaging primitives now: Messages and Methods. And each
> component
> declares: You can call these methods and send these messages.
> They are the 'Level 0' primitives of the RTT. Any transport should
> support
> these. Note that conveniently, the transport layer may implement
> messages with
> the same primitive as data ports. But we, users, don't care. We
> still have
> Data Ports to 'broadcast' our data streams and now we have Messages
> as well to
> send directly to component X.
>
> Think about it. The RTT would be highly usable if each component
> only had data
> ports and a Message/Method interface. Ask the AUTOSAR people, it's
> very close
> to what they have (and can live with).
What of Properties and Attributes? Herman briefly mentioned recently
about the aspect-oriented nature of Orocos, and that that is an oft
forgotten benefit. I agree. These aspects are huge pieces of the
productivity improvement that Orocos provides - v2.0 should *not*
forget that. This include deployment, reporting, logging, properties,
data ports, etc.
After a brief look at AUTOSAR, they appear to have done a nice job of
separating the orthogonal elements of the system: components and their
inter-connections, communications bus, deployment, and runtime
environment. A little simplification in Orocos might be in order, but
perhaps also better description or documentation or something of how
these individual domains/systems/areas all contribute to a working
system? It seems we've already got the same things ... and in general,
the things that we need.
One question, in what you saw at FOSDEM, how do AUTOSAR, et al, do
scripting and state machines? How does it differ from what Orocos
already offers?
> There's one side effect of the Message: we will need a real-time
> memory
> allocator to reserve a piece of memory for each message sent, and
> free it when
> the message is processed. Welcome TLSF. In case such a thing is not
> possible
> wanted by the user, Messages can fall back to using pre-allocated
> memory, but
> at the cost of reduced functionality (similar to what Commands can
> do today).
> Also, we'll have a MessageProcessor, which replaces and is a slimmed
> down
> version of the CommandProcessor today.
>
> So where does this leave Events? Events are of the last primitives I
> explain
> in courses because they are so complex. They don't need to be. Today
> you need
> to attach a C/C++ function to an event and optionally specify an
> EventProcessor. Depending on some this-or-thats the function is
> executed in
> this-or-the-other thread. Let's forget about that. In essence, an
> Event is a
> local thing that others like to know about: Something happened
> 'here', who
> wants to know ? Events can be changed such that you can say: If
> event 'e'
> happens, then call this Method. And you can say: if event 'e'
> happens, send me
> this Message. You can subscribe as many callbacks as you want.
> Because of the lack of this mechanism, the current Event
> implementation has a
> huge foot print. There's a lot to win here.
I argue that events must remain for state machines, particularly with
broadcasting events that affect state machines in other components.
> Do you want to allow others to raise the event ? Easy: add it to the
> Message
> or Method interface, saying: send me this Message and I'll raise the
> event, or
> call this Method and you'll raise it, respectively. But if someone
> can raise
> it, is your component's choice. That's what the event interface
> should look
> like. It's a Level 1. A transport should do no more than allowing to
> connect
> Methods and Messages (which it already supports, Level 1) to Events.
> No more.
> Even our CORBA layer could do that.
>
> The implementation of Event can benefit from a rt_malloc as well.
> Indirectly.
> Each raised Event which causes Messages to be sent out will use the
> Message's
> rt_malloc to store the event data by just sending the Message. In
> case you
> don't have/want an rt_malloc, you fall back to what events can
> roughly do
> today. But with lots of less code ( Goodbye RTT::ConnectionC, Goodbye
> RTT::EventProcessor ).
>
> And now comes the climax: Sir Command. How does he fit in the
> picture? He'll
> remain in some form, but mainly as a 'Level 2' citizen. He'll be
> composed of
> Methods, Messages and Events and will be dressed out to be no more
> than a
> wrapper, keeping related classes together or even not that.
> Replacing a
> Command with a Message hardly changes anything in the C++ side. For
> scripts,
> Commands were damn useful, but we will come up with something
> satisfactory.
> I'm sure.
>
> How does all this interface shuffling allows us to get 'towards a
> sustainable
> distributed component model'? That's because we're seriously
> lowering the
> requirements on the transport layer:
> * It only needs to implement the Level 0 primitives. How proxies and
> servers
> are built depends on the transport. You can do so manually (dlib
> like) or
> automatically (CORBA like)
> * It allows the transport to control memory better, share it between
> clients
> and clean it up at about any time.
> * The data flow changes Sylvain proposes strengthen our data flow
> model and I'm
> betting on it that it won't use CORBA as a transport. Who knows.
What might it use?
> And we are at the same time lowering the learning curve for new users:
> * You can easily explain the basic primitives: Properties=>XML,
> DataPorts=>process data, Methods/Messages=>client/server requests.
> When
> they're familiar with these, they can start playing with Events
> (which build
> on top of Method/Messages and play a role in DataPorts as well).
> And finally,
> if they'll ever need, the Convoluted Command can encompass the most
> complex
> scenarios.
I would paraphrase:
- Components (and their interconnections)
- Properties = configuration of components (XML is the medium)
- DataPorts = data communciated between components
- Methods/Messages = actions communicated between components
... and ...
- Events = broadcast changes of component state (mostly useful in
state machines)
- Attributes = temporary data state of a component
- State Machines = a components state and how it changes, what
changes it, etc.
And orthogonally
- Deployment = runtime configuration of the system (eg which
components, how each component is configured)
- Reporting = data collection
- Logging = diagnostics (typically, YMMV)
- Communication bus (currently CORBA)
> * You can more easily connect with other middleware or external
> programs.
> People with other middleware will see the opportunities for 1-to-1
> mappings or
> even implement it as a transport in the RTT.
The biggest piece missing for new users is system level examples.
Complete. Working, Up to date. Isolated, examples. A user should be
able to take one of these system examples, reimplement the "robot"
component for their specific robot, and then be able to drive it. For
those of us that learn by example, this will help massively reduce the
learning curve. A series of graduated, directed, isolated, complete
and up-to-date examples. Recent discussion about knocking out the out-
of-date and non-working examples in OCL decided to keep in these
examples, less than optimal thought they are. Simply because we have
nothing better. Orocos has great detailed examples, but very little at
a higher level demonstrating how everything goes together. Both are
needed. IMHO. Add this as a big component of v2.0 (and yes, as I
opened my mouth, I'll volunteer to help). :-)
> So this is my plan for RTT 2.0(09): not UML state machines, not
> generated
> code, not lua integration. They can wait for 3.0. Cleaning up the
> interface,
> unifying it, providing all users a smooth upgrade path, being
> similar to what
> other component frameworks offer. Setting a path out for growth.
> That's 2.0.
I think we can simplify how much writing is need to add ports, etc, to
components. There might also be some simplification in the XML
configuration files too (I like the recent reporting changes, though
I've yet to use them).
Also add real-time reporting (needs separation of data gathering from
data writing).
> These major steps won't keep us from improving what we have, There's
> a lot
> we're keeping the same. I'll setup a RTT 2.0 branch with git. By
> next FOSDEM,
> I'd like to demonstrate it. In 45 minutes. With a small ARM board.
> Running one
> component. which I conveniently control from my command line, web
> interface or
> taskbrowser on my Laptop. Or any other laptop in the audience.
Enough from me, I'll comment on more of Peter's tome later ...
S
The road to RTT 2.0: towards a sustainable distributed component
On Tue, 10 Feb 2009, Peter Soetens wrote:
> On Monday 09 February 2009 17:34:07 Herman Bruyninckx wrote:
>> On Mon, 9 Feb 2009, Peter Soetens wrote:
>>
>> [...]
>>
>>>>> * RTT has remaining usability issues. Sylvain already identified the
>>>>> short commings of data/buffer ports and proposed a solution. But any
>>>>> user wrestling with the ''Should I use an Event
>>>>> (syn/asyn)-Method-Command-DataPort'?' question only got the answer:
>>>>> ''Well, we got Events(syn/asyn)-Methods- Commands and DataPorts !'.
>>>>> It's not coherent. There are other frameworks doing a better job.
>>>>
>>>> Examples? Concrete analysis in what respects they are doing better?
>>>
>>> Simplicity is a big one. Fewer primitives which allow to build more
>>> complex primitives.
>>
>> Simplicity as you describe it here is a two-sided sword! Orocos has always
>> had different levels of "users" in mind (the framework builder, the
>> component builder, and the system builder). Each should have a "simple" set
>> of primitives to build more complex ones. But what is simple and complex is
>> different at each level! The Framework builders benefit most from the
>> suggested "2.0 redesign"; the component builders will not, since at that
>> level having only asynchronous message passing without more abstract
>> primitives is too complex! Or rather, you will immediately see reinvention
>> of the "Method", "Event" and "Command" primitives, but done poorly, and in
>> incompatible ways...
>
> I disagree: If this 2.0 redesign only benefited framework builders, I wouldn't
> propose it. It's clearly a user API thing, which in addition simplifies the
> framework implementation and middleware integration. Users may use Level 0
> primitives. They are indispensable building blocks for writing components. The
> RTT makes the abstraction to make these primitives portable and reusable.
>
So, you didn't read very well my definition of our different "users" :-) I
agree that what you propose makes writing components easier, so your
"users" are my "component builders" :-) Which makes me reiterate my worry:
pure message passing at this level is dangerous! (Because it requires a lot
of discipline not to introduce couplings between sender and receiver...)
>>> That's why I want to 'drop' the terms syn/asyn. I would only talk in
>>> terms of Methods and Messages (see below for more remarks).
>>
>> Wrong! Don't fall into the trap that hiding the complex things behind
>> easy/simple interfaces will automagically make an inherent complex activity
>> (= making high-quality, reusable, reportable, portable,...) components into
>> a simple one...
>
> I'm not dropping interfaces, I'm dropping naming conventions. In the Orocos
> Domain Specific Language, Message == asyn, Method == syn.
One should _only_ introduce a DSL when one has no other "domain independent
language" to use... Which is not the case, in this context!
>>> We're having some awkward implementations, which impose illogical
>>> restrictions on the user interface. The main thing we are missing because
>>> of no rt_malloc is that we can't work message based.
>>
>> I don't agree! Runtime memory allocation is _not_ a necessary condition for
>> asynchronous message passing! There is sufficient prior art in this respect
>> (I indicated some of it in my previous mail, such as PCI).
>
> What's in a name...
A lot!
>> [...]
>>
>>> Although the lowest level of the transport does everything message based,
>>> I don't want to hide messages from the user API, nor do I want to force
>>> the user to specify everything message based. The user actually doesn't
>>> care if there's middleware or not involved. He just needs clear
>>> semantics: If I call function A, x happens.
>>
>> These seems to be two contradictory ambitions...
>
> There seems to be a misunderstanding on your part.
So, explain your ideas better :-) If _I_ misunderstand things, than I am
quite sure new "users" will have problems too...
>>>>> There's one side effect of the Message: we will need a real-time memory
>>>>> allocator to reserve a piece of memory for each message sent, and free
>>>>> it when the message is processed. Welcome TLSF.
>>>>
>>>> Wait a bit, not so fast!!! As in every realtime system, it would be
>>>> acceptable practice to pre-allocate everything you think you need, and
>>>> not send any more messages if the pre-allocated memory is full. So, TLSF
>>>> is _one_ of the possible policies (which I support!), while the other is
>>>> the classical policy of "low-water - high-water" buffer allocation that
>>>> has proven to work in highly coupled embedded stuff (such as device
>>>> drivers with PCI support, and many others). Hence: okay for the
>>>> introduction of TLSF, but not as the only option.
>>>
>>> Yes. And the other option is what I call the 'fall-back'.
>>
>> No, they should get _equal_ status! With some explanations in the
>> documentation about the most appropriate use cases (and usage methods) for
>> both.
>
> If they were completely equal, we wouldn't need TLSF in the first place.
Please, read what I write: both options should have equal _status_ in
Orocos. I never claimed that they provide equal _technical solutions_...
> Look,
> we have the ORO_EMBEDDED option, this breaks many applications because it
> turns many features off. People who turn this on nevertheless, know what
> they're doing, or find out real quickly :-). The change I'm proposing has equal
> effects. If you turn TLSF off, you'll get reduced functions.
Which is fine. In the long term, this kind of flexibility is only workable
if their is toolchain support for it, in order (i) to keep things
consistent, and (ii) to inform users about the consequences of their
choices.
>>> Just to be clear: although the lowest levels of the middleware (Level -1)
>>> use a message based transport, At the user API level, there are
>>> RTT::Message objects as well (which will probably map 1:1 to a middleware
>>> message, but this is hidden).
>>
>> Be careful about mixing two levels in the same API... It would be
>> worthwhile having multiple APIs, as in most software stacks.
>
> I start having remorse about the use of the word 'Level'... You should not
> understand level as 'abstraction level', but as entry level (L0), intermediate
> level (L1) and advanced level (L2) *user API*. I'm mainly discussing user
> interfaces here, and only a few details about the hidden (L-1) transport
> level.
Again: "What's in a name?" is not a thing we should tread on lightly...
>> Go for it!
>
> I first have to get past the UML deamon before I can enter the next level of
> the Implementation Game.
I am not putting any pressure :-) Enjoy the game...
Herman
The road to RTT 2.0: towards a sustainable distributed component
On Sun, Feb 08, 2009 at 09:35:10PM -0500, S Roderick wrote:
> > * RTT interoperates badly with other software, for example, any
> > external
> > process needs to go through a convoluted CORBA layer. There are also
> > no tools
> > that could ease the job (except the ctaskbrowser), for example some
> > small
> > shell commands that can query/change a component.
>
> I love the Corba integration! It's butt easy ... problem is, we
> provide no application level examples of how to use.
>
> How much simpler can a GUI be than
>
> find deployer ** close to a one-liner **
> find peer ** ditto **
> local_method = peer.getMethd()
> ...
> local_method(a,b,c)
I agree that this simplicity is nice, but it also "locks" one into
CORBA. The slimmer the interface required by the primitives are, the
easier it is to plug-in replacements. For example consider the
following. From a robustness POV the concepts of threads is quite
fragile: it pretty easy to corrupt memory, segfault etc. and bring
down the whole application due to one erronous thread. Processes in
contrast, are much more robust and can continue to run even if other
cooperating ones crash. But how could we do this in OROCOS,
e.g. having a set of local components partioned at process
granularity? I guess it would be impossible to do without CORBA. But
if the primitives are only concepts like "send/receive" message, it
becomes trivial to choose fast and lightweight mechanism such as Unix
domain sockets or Message queues instead.
> I think we can simplify how much writing is need to add ports, etc, to
> components. There might also be some simplification in the XML
I agree! This will certainly be tackled by code generation, but OTOH,
the simpler the generated code the better.
> configuration files too (I like the recent reporting changes, though
> I've yet to use them).
Agreed too. I dare ask the question if XML is the best choice for
configuration files in the first place? There is an interesting
article by the ANTLR author on this:
http://www.ibm.com/developerworks/library/x-sbxml.html
Best regards
Markus
The road to RTT 2.0: towards a sustainable distributed component
On Monday 09 February 2009 03:35:10 S Roderick wrote:
> Holy crap, that's the longest email I've seen on this list ...
>
> On Feb 8, 2009, at 18:26 , Peter Soetens wrote:
> > * RTT interoperates badly with other software, for example, any
> > external
> > process needs to go through a convoluted CORBA layer. There are also
> > no tools
> > that could ease the job (except the ctaskbrowser), for example some
> > small
> > shell commands that can query/change a component.
>
> I love the Corba integration! It's butt easy ... problem is, we
> provide no application level examples of how to use.
>
> How much simpler can a GUI be than
>
> find deployer ** close to a one-liner **
> find peer ** ditto **
> local_method = peer.getMethd()
> ...
> local_method(a,b,c)
Whatever you're showing, it's not CORBA, its RTT. Don't get me wrong, CORBA
will be an integral part of RTT 2.0. But I hope it will: 1. make the CORBA
implementation easier/robuster and 2. make alternatives easier to implement.
>
> How much more do you want? The code in the GUI looks exactly like code
> in local components. Sure, the semantics of timing, latency, etc, are
> different, but it's going across IP one way or the other. Deal with it.
>
> Now, writing new CORBA plugins is like being in hell or purgatory ...
> that needs serious work ... my sanity will never be the same after
> doing 2 of those ... and I've more to come ... :-(
Ack.
>
> > * RTT has remaining usability issues. Sylvain already identified the
> > short
> > commings of data/buffer ports and proposed a solution. But any user
> > wrestling
> > with the ''Should I use an Event (syn/asyn)-Method-Command-DataPort'?'
> > question only got the answer: ''Well, we got Events(syn/asyn)-
> > Methods-
> > Commands and DataPorts !'. It's not coherent. There are other
> > frameworks doing
> > a better job. We can do a far better job.
>
> There are a lot of pieces, agreed. It takes a long time to come up the
> learning curve ... though each piece does have it's place.
What I'm trying to find is a kind of hierarchy in the execution flow interface.
Level 0 are the basic features, learn to use that, than play with Level 1 and
only then Level 2. Most users will end up with Level 0 and 1, imho.
>
> > * RTT has issues with its current distribution implementation:
> > programs can be
> > constructed as such that they cause mem leaks at the remote side,
> > Events never
> > got into the CORBA interface (there is a reason for that), and our
> > data ports
> > over CORBA are equally weak as the C++ implementation.
>
> How are data ports weak?
Because they have to strong coupling :-) See Sylvain's detailed analysis of
commit message:
http://github.com/doudou/orocos-
rtt/commit/dc1947c8c1bdace90cf0a3aa2047ad248619e76b
That text belongs in the wiki really...
>
> > * And then there are also the untaken opportunities to reduce RTT &
> > component
> > code size drastically and remove complex features.
> >
> > What does this all mean ? Start over ? Not at all. I'll defend that
> > with small
>
> You start over, and I sail all the way to Brussels and shoot your
> sorry behind!
I wouldn't want to be in any paint-ball game with any Orocos user, anytime.
>
> > changes (of which some break backwards compatibility) we can solve
> > all the
> > above in one swift sweep. Unfortunately, it all starts with adding a
> > new (but
> > optionally optional) feature: rt_malloc, aka TLSF. Yes you saw me
> > write I
> > would remove features, by adding a feature (well, actually two). I can
> > explain.
>
> TLSF would help greatly in some situations. Would also open the door
> for abuse, but people have to learn somewhere.
I would at first only use it for internal management (messages and data ports).
Any user can already today add his own TLSF heap.
>
> Events are fantastic when combined with state machines!! Thank you,
> thank you, thank you ... there, now you have 4 thank yous for
> Events! ;-) I *regularly* use events to inter-connect state
> machines. It makes this trivial.
Well, that one thank you was actually already from you. Events will work
exactly as they do work now in scripting State Machines. I want them to be
qually usable in C++.
> >
> > Think about it. The RTT would be highly usable if each component
> > only had data
> > ports and a Message/Method interface. Ask the AUTOSAR people, it's
> > very close
> > to what they have (and can live with).
>
> What of Properties and Attributes?
I meant to write: .. would already be highly usable... I'm not proposing to
drop properties, attributes, events, scripting etc. Just stream-lining our
execution interface (me) and data flow interface (Sylvain).
> Herman briefly mentioned recently
> about the aspect-oriented nature of Orocos, and that that is an oft
> forgotten benefit. I agree. These aspects are huge pieces of the
> productivity improvement that Orocos provides - v2.0 should *not*
> forget that. This include deployment, reporting, logging, properties,
> data ports, etc.
>
> After a brief look at AUTOSAR, they appear to have done a nice job of
> separating the orthogonal elements of the system: components and their
> inter-connections, communications bus, deployment, and runtime
> environment. A little simplification in Orocos might be in order, but
> perhaps also better description or documentation or something of how
> these individual domains/systems/areas all contribute to a working
> system? It seems we've already got the same things ... and in general,
> the things that we need.
>
> One question, in what you saw at FOSDEM, how do AUTOSAR, et al, do
> scripting and state machines? How does it differ from what Orocos
> already offers?
AUTOSAR was not presented at FOSDEM. AUTOSAR is stronger in platform
abstraction (which adds limitations and overhead), but has less features than
Orocos. They only have Level 1 + data ports. AUTOSAR is for the very embedded.
>
> I argue that events must remain for state machines, particularly with
> broadcasting events that affect state machines in other components.
100% sure.
> > How does all this interface shuffling allows us to get 'towards a
> > sustainable
> > distributed component model'? That's because we're seriously
> > lowering the
> > requirements on the transport layer:
> > * It only needs to implement the Level 0 primitives. How proxies and
> > servers
> > are built depends on the transport. You can do so manually (dlib
> > like) or
> > automatically (CORBA like)
> > * It allows the transport to control memory better, share it between
> > clients
> > and clean it up at about any time.
> > * The data flow changes Sylvain proposes strengthen our data flow
> > model and I'm
> > betting on it that it won't use CORBA as a transport. Who knows.
>
> What might it use?
Why not CORBA ? Because CORBA is a request-response architecture, which blocks
when sending. One-ways might be a solution, but in the end, the CORBA folks
themselves defend to set data streams up 'out of band'. So the connection is
made using CORBA, the data stream transport is done outside CORBA. I
believe/hope UDP based Data Flow will be sufficient as a default implementation.
>
> > And we are at the same time lowering the learning curve for new users:
> > * You can easily explain the basic primitives: Properties=>XML,
> > DataPorts=>process data, Methods/Messages=>client/server requests.
> > When
> > they're familiar with these, they can start playing with Events
> > (which build
> > on top of Method/Messages and play a role in DataPorts as well).
> > And finally,
> > if they'll ever need, the Convoluted Command can encompass the most
> > complex
> > scenarios.
>
> I would paraphrase:
> - Components (and their interconnections)
> - Properties = configuration of components (XML is the medium)
> - DataPorts = data communciated between components
> - Methods/Messages = actions communicated between components
> ... and ...
> - Events = broadcast changes of component state (mostly useful in
> state machines)
> - Attributes = temporary data state of a component
> - State Machines = a components state and how it changes, what
> changes it, etc.
>
> And orthogonally
> - Deployment = runtime configuration of the system (eg which
> components, how each component is configured)
> - Reporting = data collection
> - Logging = diagnostics (typically, YMMV)
> - Communication bus (currently CORBA)
You got it.
>
> > * You can more easily connect with other middleware or external
> > programs.
> > People with other middleware will see the opportunities for 1-to-1
> > mappings or
> > even implement it as a transport in the RTT.
>
> The biggest piece missing for new users is system level examples.
> Complete. Working, Up to date. Isolated, examples. A user should be
> able to take one of these system examples, reimplement the "robot"
> component for their specific robot, and then be able to drive it. For
> those of us that learn by example, this will help massively reduce the
> learning curve. A series of graduated, directed, isolated, complete
> and up-to-date examples. Recent discussion about knocking out the out-
> of-date and non-working examples in OCL decided to keep in these
> examples, less than optimal thought they are. Simply because we have
> nothing better. Orocos has great detailed examples, but very little at
> a higher level demonstrating how everything goes together. Both are
> needed. IMHO. Add this as a big component of v2.0 (and yes, as I
> opened my mouth, I'll volunteer to help). :-)
The wiki is up and running.
>
> I think we can simplify how much writing is need to add ports, etc, to
> components. There might also be some simplification in the XML
> configuration files too (I like the recent reporting changes, though
> I've yet to use them).
>
> Also add real-time reporting (needs separation of data gathering from
> data writing).
We'll ensemble the targets and use cases (old vs new) on Wiki pages.
Just to sum up what I was aiming for solving:
* Solve descripancies between Method-Command-Event and unify the API
* have a lighter weight primitive than commands
* have events go over CORBA
* Fix memory management in distributed setups
* The data flow patch of Sylvain will need a major release of RTT (2.0), I
believe these changes go along with his.
Peter
[note]
On Monday 09 February 2009 00:26:25 Peter Soetens wrote:
...
Do I need to say: Feel free to chop this text to pieces and put it in flames.
I'll calmly listen to the crackling and warmly reply. You guys need to drive
this.