[Bug 728] New: Add real-time memory allocation support

Submitted by snrkiwi on Thu, 2009-11-05 18:28

RTT-dev

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

Summary: Add real-time memory allocation support
Product: OCL
Version: trunk
Platform: All
OS/Version: All
Status: NEW
Severity: normal
Priority: P3
Component: Other
AssignedTo: orocos-dev [..] ...
ReportedBy: kiwi [dot] net [..] ...
CC: orocos-dev [..] ...
Estimated Hours: 0.0

Integrate TLSF into OCL

[Bug 728] Add real-time memory allocation support

Submitted by peter on Mon, 2010-05-03 10:44.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

--- Comment #15 from Peter Soetens <peter [..] ...> 2010-05-03 12:41:28 ---
I'm picking this one up again. I actually overlooked this bug report. I'll try
to apply the patches on 1.x/2.0 lines and see what it gives... Worth doing ?

Peter

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Mon, 2010-05-03 11:16.

On May 3, 2010, at 06:41 , Peter Soetens wrote:

> https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728
>
>
>
>
>
> --- Comment #15 from Peter Soetens <peter [..] ...> 2010-05-03 12:41:28 ---
> I'm picking this one up again. I actually overlooked this bug report. I'll try
> to apply the patches on 1.x/2.0 lines and see what it gives... Worth doing ?
>
> Peter

Yes, worth doing IMHO. I've been using this for quite some time, and have had no issues on macosx nor gnulinux.

I did start trying to extend the type plugin to support use of rt-strings in scripts and state machines, but hit some parser issues that I didn't solve (string had precedence over rt-string, IIRC). I will return to this soon.

You might check out my release-rtt-rtalloc branch on github, as it has the latest patches (including dealing with some boost v1.42 madness)

Stephen

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Fri, 2009-11-13 13:44.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

--- Comment #13 from S Roderick <kiwi [dot] net [..] ...> 2009-11-13 14:39:47 ---
Created an attachment (id=566)
--> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=566)
Add command line options to deployers to set size of real-time memory pool

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Fri, 2009-11-13 13:44.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

--- Comment #14 from S Roderick <kiwi [dot] net [..] ...> 2009-11-13 14:40:05 ---
Created an attachment (id=567)
--> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=567)
Add toolkit and transport plugins for rtalloc types

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Fri, 2009-11-13 13:40.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

S Roderick <kiwi [dot] net [..] ...> changed:

--- Comment #12 from S Roderick <kiwi [dot] net [..] ...> 2009-11-13 14:39:19 ---
Created an attachment (id=565)
--> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=565)
Provide real-time allocatable dynamically-sized string using TLSF embedded in
RTT

Uses TLSF embedded into RTT v1.x. Requires patches from #733.

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Fri, 2009-11-13 13:32.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

S Roderick <kiwi [dot] net [..] ...> changed:

What |Removed |Added
----------------------------------------------------------------------------
Depends on| |733

--- Comment #11 from S Roderick <kiwi [dot] net [..] ...> 2009-11-13 14:28:31 ---
Requires #733

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Tue, 2009-11-10 17:08.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

--- Comment #10 from S Roderick <kiwi [dot] net [..] ...> 2009-11-10 18:07:12 ---
Created an attachment (id=556)
--> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=556)
Add missing header files needed for Ubuntu Jaunty and Koala

Tested on Heron, Jauntaloupe, Koala and Snow Leopard ... feels like I'm at the
damn zoo! :-)

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Tue, 2009-11-10 17:08.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

S Roderick <kiwi [dot] net [..] ...> changed:

--- Comment #9 from S Roderick <kiwi [dot] net [..] ...> 2009-11-10 18:06:08 ---
Created an attachment (id=555)
--> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=555)
Modify deployers to support memory allocation size on command line

[Bug 728] Add real-time memory allocation support

Submitted by peter on Tue, 2009-11-10 16:24.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

--- Comment #8 from Peter Soetens <peter [..] ...> 2009-11-10 17:15:41 ---
(In reply to comment #7)
> Created an attachment (id=553)
--> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=553)
> Sample deployer modifications to support memory allocation size on command line
>
> I'm looking for feedback on the following, before proposing a complete patch
> that modifies all deployers:
>
> - Does this overall approach work with the intent to embed TLSF into RTT?

RTT will adopt boost::po as well to parse some command line options, so your
code
could migrate to RTT::os to setup TLSF. Same thing can be said for setting up
the RTT log levels btw.

>
> - Is the custom type in deployer_funcs.hpp acceptable? This is the only way that
> I could find to use boost's custom validators, to allow memory size to be
> specified as, for example, 10240, 4k, 32M, etc. It also means we aren't passing
> yet another parameter into deployerParseCmdLine(), and are instead using the
> functionality already provided by boost::program_options.

I see no problem at all. You got to do what you got to do. Looks clean to me.

Peter

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Tue, 2009-11-10 16:44.

On Nov 10, 2009, at 11:15 , Peter Soetens wrote:

> https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728
>
>
>
>
>
> --- Comment #8 from Peter Soetens <peter [..] ...>
> 2009-11-10 17:15:41 ---
> (In reply to comment #7)
>> Created an attachment (id=553)
> --> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=553)
>> Sample deployer modifications to support memory allocation size on
>> command line
>>
>> I'm looking for feedback on the following, before proposing a
>> complete patch
>> that modifies all deployers:
>>
>> - Does this overall approach work with the intent to embed TLSF
>> into RTT?
>
> RTT will adopt boost::po as well to parse some command line options,
> so your
> code
> could migrate to RTT::os to setup TLSF. Same thing can be said for
> setting up
> the RTT log levels btw.

Good. How do you intend to integrate boost::po into RTT? Is the
deployer moving into RTT, as you've previously mentioned might happen?

>> - Is the custom type in deployer_funcs.hpp acceptable? This is the
>> only way that
>> I could find to use boost's custom validators, to allow memory size
>> to be
>> specified as, for example, 10240, 4k, 32M, etc. It also means we
>> aren't passing
>> yet another parameter into deployerParseCmdLine(), and are instead
>> using the
>> functionality already provided by boost::program_options.
>
> I see no problem at all. You got to do what you got to do. Looks
> clean to me.

Ok, will provide full patch shortly.

Also, I think this overall approach will result in less OCL code in
the long run, as we can get rid of nearly all of deployer-funcs by
using some other boost::po functionality instead of our custom code.
I'll look at it another time ...

Stephen

[Bug 728] Add real-time memory allocation support

Submitted by peter on Tue, 2009-11-10 20:16.

On Tue, Nov 10, 2009 at 17:37, S Roderick <kiwi [dot] net [..] ...> wrote:
> On Nov 10, 2009, at 11:15 , Peter Soetens wrote:
>
>> https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728
>>
>>
>>
>>
>>
>> --- Comment #8 from Peter Soetens <peter [..] ...> 2009-11-10
>> 17:15:41 ---
>> (In reply to comment #7)
>>>
>>> Created an attachment (id=553)
>>
>> --> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=553)
>>>
>>> Sample deployer modifications to support memory allocation size on
>>> command line
>>>
>>> I'm looking for feedback on the following, before proposing a complete
>>> patch
>>> that modifies all deployers:
>>>
>>> - Does this overall approach work with the intent to embed TLSF into RTT?
>>
>> RTT will adopt boost::po as well to parse some command line options, so
>> your
>> code
>> could migrate to RTT::os to setup TLSF. Same thing can be said for setting
>> up
>> the RTT log levels btw.
>
> Good. How do you intend to integrate boost::po into RTT? Is the deployer
> moving into RTT, as you've previously mentioned might happen?

RTT won't include components, it remains a library. It will use the
boost::po to interprete the arguments that go into ORO_main(). If you
were referring to the plugin loading code moving into RTT, that's more
for OS abstraction purposes than application functionality as in
'deployer'.

>
>>> - Is the custom type in deployer_funcs.hpp acceptable? This is the only
>>> way that
>>> I could find to use boost's custom validators, to allow memory size to be
>>> specified as, for example, 10240, 4k, 32M, etc. It also means we aren't
>>> passing
>>> yet another parameter into deployerParseCmdLine(), and are instead using
>>> the
>>> functionality already provided by boost::program_options.
>>
>> I see no problem at all. You got to do what you got to do. Looks clean to
>> me.
>
> Ok, will provide full patch shortly.
>
> Also, I think this overall approach will result in less OCL code in the long
> run, as we can get rid of nearly all of deployer-funcs by using some other
> boost::po functionality instead of our custom code. I'll look at it another
> time ...

Do you have an idea on which boost version you're depending for all this ?

Thanks for keeping all this in bug reports, there would be no way of
tracking your stream of patches otherwise :-)

Peter

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Tue, 2009-11-10 20:28.

On Nov 10, 2009, at 15:14 , Peter Soetens wrote:

> On Tue, Nov 10, 2009 at 17:37, S Roderick <kiwi [dot] net [..] ...> wrote:
>> On Nov 10, 2009, at 11:15 , Peter Soetens wrote:
>>
>>> https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728
>>>
>>>
>>>
>>>
>>>
>>> --- Comment #8 from Peter Soetens <peter [..] ...>
>>> 2009-11-10
>>> 17:15:41 ---
>>> (In reply to comment #7)
>>>>
>>>> Created an attachment (id=553)
>>>
>>> --> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=553)
>>>>
>>>> Sample deployer modifications to support memory allocation size on
>>>> command line
>>>>
>>>> I'm looking for feedback on the following, before proposing a
>>>> complete
>>>> patch
>>>> that modifies all deployers:
>>>>
>>>> - Does this overall approach work with the intent to embed TLSF
>>>> into RTT?
>>>
>>> RTT will adopt boost::po as well to parse some command line
>>> options, so
>>> your
>>> code
>>> could migrate to RTT::os to setup TLSF. Same thing can be said for
>>> setting
>>> up
>>> the RTT log levels btw.
>>
>> Good. How do you intend to integrate boost::po into RTT? Is the
>> deployer
>> moving into RTT, as you've previously mentioned might happen?
>
> RTT won't include components, it remains a library. It will use the
> boost::po to interprete the arguments that go into ORO_main(). If you
> were referring to the plugin loading code moving into RTT, that's more
> for OS abstraction purposes than application functionality as in
> 'deployer'.

Understood.

>>>> - Is the custom type in deployer_funcs.hpp acceptable? This is
>>>> the only
>>>> way that
>>>> I could find to use boost's custom validators, to allow memory
>>>> size to be
>>>> specified as, for example, 10240, 4k, 32M, etc. It also means we
>>>> aren't
>>>> passing
>>>> yet another parameter into deployerParseCmdLine(), and are
>>>> instead using
>>>> the
>>>> functionality already provided by boost::program_options.
>>>
>>> I see no problem at all. You got to do what you got to do. Looks
>>> clean to
>>> me.
>>
>> Ok, will provide full patch shortly.
>>
>> Also, I think this overall approach will result in less OCL code in
>> the long
>> run, as we can get rid of nearly all of deployer-funcs by using
>> some other
>> boost::po functionality instead of our custom code. I'll look at it
>> another
>> time ...
>
> Do you have an idea on which boost version you're depending for all
> this ?

Hence the "zoo" of tests earlier. Hardy is 1.34.1, SL is 1.40.0, and
Koala is 1.40.0. I figured if I bracketed these, then I was probably
in pretty good shape. Any others you can think of for first-round
testing? Didn't Ruben offer to do all testing for the wonderful
Debian ... or was it another adjective besides "wonderful" ... ;-)

> Thanks for keeping all this in bug reports, there would be no way of
> tracking your stream of patches otherwise :-)

I am _trying_ to make it easy for you, so that this stuff doesn't end
up just in my own repo! Smack me if it becomes hard for you ... :-)
S

[Bug 728] Add real-time memory allocation support

Submitted by Ruben Smits on Wed, 2009-11-11 20:44.

On Tue, Nov 10, 2009 at 9:19 PM, Stephen Roderick <kiwi [dot] net [..] ...> wrote:
> On Nov 10, 2009, at 15:14 , Peter Soetens wrote:
>
>> On Tue, Nov 10, 2009 at 17:37, S Roderick <kiwi [dot] net [..] ...> wrote:
>>> On Nov 10, 2009, at 11:15 , Peter Soetens wrote:
>>>
>>>> https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --- Comment #8 from Peter Soetens <peter [..] ...>
>>>> 2009-11-10
>>>> 17:15:41 ---
>>>> (In reply to comment #7)
>>>>>
>>>>> Created an attachment (id=553)
>>>>
>>>> --> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=553)
>>>>>
>>>>> Sample deployer modifications to support memory allocation size on
>>>>> command line
>>>>>
>>>>> I'm looking for feedback on the following, before proposing a
>>>>> complete
>>>>> patch
>>>>> that modifies all deployers:
>>>>>
>>>>> - Does this overall approach work with the intent to embed TLSF
>>>>> into RTT?
>>>>
>>>> RTT will adopt boost::po as well to parse some command line
>>>> options, so
>>>> your
>>>> code
>>>> could migrate to RTT::os to setup TLSF. Same thing can be said for
>>>> setting
>>>> up
>>>> the RTT log levels btw.
>>>
>>> Good. How do you intend to integrate boost::po into RTT? Is the
>>> deployer
>>> moving into RTT, as you've previously mentioned might happen?
>>
>> RTT won't include components, it remains a library. It will use the
>> boost::po to interprete the arguments that go into ORO_main(). If you
>> were referring to the plugin loading code moving into RTT, that's more
>> for OS abstraction purposes than application functionality as in
>> 'deployer'.
>
> Understood.
>
>>>>> - Is the custom type in deployer_funcs.hpp acceptable? This is
>>>>> the only
>>>>> way that
>>>>> I could find to use boost's custom validators, to allow memory
>>>>> size to be
>>>>> specified as, for example, 10240, 4k, 32M, etc. It also means we
>>>>> aren't
>>>>> passing
>>>>> yet another parameter into deployerParseCmdLine(), and are
>>>>> instead using
>>>>> the
>>>>> functionality already provided by boost::program_options.
>>>>
>>>> I see no problem at all. You got to do what you got to do. Looks
>>>> clean to
>>>> me.
>>>
>>> Ok, will provide full patch shortly.
>>>
>>> Also, I think this overall approach will result in less OCL code in
>>> the long
>>> run, as we can get rid of nearly all of deployer-funcs by using
>>> some other
>>> boost::po functionality instead of our custom code. I'll look at it
>>> another
>>> time ...
>>
>> Do you have an idea on which boost version you're depending for all
>> this ?
>
> Hence the "zoo" of tests earlier. Hardy is 1.34.1, SL is 1.40.0, and
> Koala is 1.40.0. I figured if I bracketed these, then I was probably
> in pretty good shape. Any others you can think of for first-round
> testing? Didn't Ruben offer to do all testing for the wonderful
> Debian ... or was it another adjective besides "wonderful" ... ;-)

What is this "Debian" you speak of ;), we are currently setting up a
build farm, to test builds on all kinds of platforms. Just a matter of
time to set up the configuration matrix.

Ruben

>> Thanks for keeping all this in bug reports, there would be no way of
>> tracking your stream of patches otherwise :-)
>
> I am _trying_ to make it easy for you, so that this stuff doesn't end
> up just in my own repo! Smack me if it becomes hard for you ... :-)
> S
>
> --
> Orocos-Dev mailing list
> Orocos-Dev [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Tue, 2009-11-10 15:48.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

--- Comment #7 from S Roderick <kiwi [dot] net [..] ...> 2009-11-10 16:47:14 ---
Created an attachment (id=553)
--> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=553)
Sample deployer modifications to support memory allocation size on command line

I'm looking for feedback on the following, before proposing a complete patch
that modifies all deployers:

- Does this overall approach work with the intent to embed TLSF into RTT?

- Is the custom type in deployer_funcs.hpp acceptable? This is the only way
that I could find to use boost's custom validators, to allow memory size to be
specified as, for example, 10240, 4k, 32M, etc. It also means we aren't passing
yet another parameter into deployerParseCmdLine(), and are instead using the
functionality already provided by boost::program_options.

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Thu, 2009-11-05 19:24.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

--- Comment #6 from S Roderick <kiwi [dot] net [..] ...> 2009-11-05 20:22:25 ---
Created an attachment (id=549)
--> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=549)
Corrected patch to add plugins

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Thu, 2009-11-05 19:24.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

S Roderick <kiwi [dot] net [..] ...> changed:

--- Comment #5 from S Roderick <kiwi [dot] net [..] ...> 2009-11-05 20:22:05 ---
Created an attachment (id=548)
--> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=548)
Corrected patch to add RT memory allocation support

It is _so_ easy to screw yourself in git ... :-( ... sorry for the noise.

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Thu, 2009-11-05 18:56.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

--- Comment #4 from S Roderick <kiwi [dot] net [..] ...> 2009-11-05 19:51:51 ---
Created an attachment (id=547)
--> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=547)
Add TLSF cmake find package

Missing from previous patches

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Thu, 2009-11-05 18:32.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

S Roderick <kiwi [dot] net [..] ...> changed:

--- Comment #1 from S Roderick <kiwi [dot] net [..] ...> 2009-11-05 19:28:10 ---
Created an attachment (id=544)
--> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=544)
Patch TLSF for use with Orocos

Allows build with CMake, and inclusion of TLSF in C++ files.

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Thu, 2009-11-05 18:32.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

--- Comment #2 from S Roderick <kiwi [dot] net [..] ...> 2009-11-05 19:28:58 ---
Created an attachment (id=545)
--> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=545)
Add real-time memory support

Add OCL::String as a real-time allocatable string. Requires TLSF

[Bug 728] Add real-time memory allocation support

Submitted by snrkiwi on Thu, 2009-11-05 18:32.

https://www.fmtc.be/bugzilla/orocos/show_bug.cgi?id=728

--- Comment #3 from S Roderick <kiwi [dot] net [..] ...> 2009-11-05 19:29:18 ---
Created an attachment (id=546)
--> (https://www.fmtc.be/bugzilla/orocos/attachment.cgi?id=546)
Add plugins for OCL::String