On May 23, 2011, at 05:55 , Theo de Vries wrote:
> I have been searching a bit and saw that some people (Ruben Smits,
> Philippe Hamelin?) used RTT with Linux and the rt preempt patch. We
> are planning to do the same.
> As far as I understand from this page
> (https://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO), we have to do
> the following:
>
> 1. Setting a real time scheduling policy and priority.
RTT takes care of this for you. Set an activity to ORO_SCHED_RT and the period and priority of your choice.
> 2. Locking memory so that page faults caused by virtual memory will
> not undermine deterministic behavior
Well, actually locking prevents paging at all ... which stops the non-deterministic behaviour.
> 3. Pre-faulting the stack, so that a future stack fault will not
> undermine deterministic behavior
We actually never do this (as its actually quite hard to do), though you are supposed to. We also never explicitly pre-fault any large data structures (as you are supposed to), as given that most of our data structures are complex objects their constructors take care of that for us. Despite this, we still run in real-time.
All the above are required for real-time performance, however, there are other necessary things you must take care of yourself, for examle
- don't dynamically allocate memory (unless you are using TLSF) once you are into the real-time part of your application
- don't use RTT::Logger (its not realtime)
- don't write to disk in a real-time component.
- etc.
We monitor our "realtime-ness" (in part) by tracking major and minor page faults on either a per-thread or per-process basis (which ever makes sense given the application). At least that way you have data telling you whether you stayed in real-time or not. NB this data isn't sufficient to tell the whole story, but it captures most problem cases.
HTH
S
orocos rtt with rt preempt
On Mon, May 23, 2011 at 01:15:31PM +0200, S Roderick wrote:
> On May 23, 2011, at 05:55 , Theo de Vries wrote:
>
> > I have been searching a bit and saw that some people (Ruben Smits,
> > Philippe Hamelin?) used RTT with Linux and the rt preempt patch. We
> > are planning to do the same.
> > As far as I understand from this page
> > (https://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO), we have to do
> > the following:
> >
> > 1. Setting a real time scheduling policy and priority.
>
> RTT takes care of this for you. Set an activity to ORO_SCHED_RT and the period and priority of your choice.
>
> > 2. Locking memory so that page faults caused by virtual memory will
> > not undermine deterministic behavior
>
> Well, actually locking prevents paging at all ... which stops the non-deterministic behaviour.
Do you mlockall in configureHook? (This should be be a deployment
property, shouldn't it)
Markus
orocos rtt with rt preempt
On May 23, 2011, at 07:25 , Markus Klotzbuecher wrote:
> On Mon, May 23, 2011 at 01:15:31PM +0200, S Roderick wrote:
>> On May 23, 2011, at 05:55 , Theo de Vries wrote:
>>
>>> I have been searching a bit and saw that some people (Ruben Smits,
>>> Philippe Hamelin?) used RTT with Linux and the rt preempt patch. We
>>> are planning to do the same.
>>> As far as I understand from this page
>>> (https://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO), we have to do
>>> the following:
>>>
>>> 1. Setting a real time scheduling policy and priority.
>>
>> RTT takes care of this for you. Set an activity to ORO_SCHED_RT and the period and priority of your choice.
>>
>>> 2. Locking memory so that page faults caused by virtual memory will
>>> not undermine deterministic behavior
>>
>> Well, actually locking prevents paging at all ... which stops the non-deterministic behaviour.
>
> Do you mlockall in configureHook? (This should be be a deployment
> property, shouldn't it)
>
> Markus
And last but not least, you need security privileges to do this on modern Linux. See /etc/security/limits.conf on all distro's we use (to avoid having to run as root/sudo).
S
orocos rtt with rt preempt
On Mon, May 23, 2011 at 01:36:45PM +0200, Stephen Roderick wrote:
> On May 23, 2011, at 07:25 , Markus Klotzbuecher wrote:
>
> > On Mon, May 23, 2011 at 01:15:31PM +0200, S Roderick wrote:
> >> On May 23, 2011, at 05:55 , Theo de Vries wrote:
> >>
> >>> I have been searching a bit and saw that some people (Ruben Smits,
> >>> Philippe Hamelin?) used RTT with Linux and the rt preempt patch. We
> >>> are planning to do the same.
> >>> As far as I understand from this page
> >>> (https://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO), we have to do
> >>> the following:
> >>>
> >>> 1. Setting a real time scheduling policy and priority.
> >>
> >> RTT takes care of this for you. Set an activity to ORO_SCHED_RT and the period and priority of your choice.
> >>
> >>> 2. Locking memory so that page faults caused by virtual memory will
> >>> not undermine deterministic behavior
> >>
> >> Well, actually locking prevents paging at all ... which stops the non-deterministic behaviour.
> >
> > Do you mlockall in configureHook? (This should be be a deployment
> > property, shouldn't it)
> >
> > Markus
>
> And last but not least, you need security privileges to do this on
> modern Linux. See /etc/security/limits.conf on all distro's we use
> (to avoid having to run as root/sudo).
Or use capabilities(7) to grant these on a per file basis.
Markus
orocos rtt with rt preempt
On May 23, 2011, at 07:25 , Markus Klotzbuecher wrote:
> On Mon, May 23, 2011 at 01:15:31PM +0200, S Roderick wrote:
>> On May 23, 2011, at 05:55 , Theo de Vries wrote:
>>
>>> I have been searching a bit and saw that some people (Ruben Smits,
>>> Philippe Hamelin?) used RTT with Linux and the rt preempt patch. We
>>> are planning to do the same.
>>> As far as I understand from this page
>>> (https://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO), we have to do
>>> the following:
>>>
>>> 1. Setting a real time scheduling policy and priority.
>>
>> RTT takes care of this for you. Set an activity to ORO_SCHED_RT and the period and priority of your choice.
>>
>>> 2. Locking memory so that page faults caused by virtual memory will
>>> not undermine deterministic behavior
>>
>> Well, actually locking prevents paging at all ... which stops the non-deterministic behaviour.
>
> Do you mlockall in configureHook? (This should be be a deployment
> property, shouldn't it)
Yes, and yes it really should be.
Also, remember that mlockall() alone isn't sufficient - you also have to set some malloc options (see the RT Wiki page referenced above, IIRC).
S
orocos rtt with rt preempt
On Mon, May 23, 2011 at 01:35:25PM +0200, Stephen Roderick wrote:
> On May 23, 2011, at 07:25 , Markus Klotzbuecher wrote:
>
> > On Mon, May 23, 2011 at 01:15:31PM +0200, S Roderick wrote:
> >> On May 23, 2011, at 05:55 , Theo de Vries wrote:
> >>
> >>> I have been searching a bit and saw that some people (Ruben Smits,
> >>> Philippe Hamelin?) used RTT with Linux and the rt preempt patch. We
> >>> are planning to do the same.
> >>> As far as I understand from this page
> >>> (https://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO), we have to do
> >>> the following:
> >>>
> >>> 1. Setting a real time scheduling policy and priority.
> >>
> >> RTT takes care of this for you. Set an activity to ORO_SCHED_RT and the period and priority of your choice.
> >>
> >>> 2. Locking memory so that page faults caused by virtual memory will
> >>> not undermine deterministic behavior
> >>
> >> Well, actually locking prevents paging at all ... which stops the non-deterministic behaviour.
> >
> > Do you mlockall in configureHook? (This should be be a deployment
> > property, shouldn't it)
>
> Yes, and yes it really should be.
Ok.
> Also, remember that mlockall() alone isn't sufficient - you also
> have to set some malloc options (see the RT Wiki page referenced
> above, IIRC).
Not sure which options you mean here...
Markus
orocos rtt with rt preempt
On May 23, 2011, at 07:41 , Markus Klotzbuecher wrote:
> On Mon, May 23, 2011 at 01:35:25PM +0200, Stephen Roderick wrote:
>> On May 23, 2011, at 07:25 , Markus Klotzbuecher wrote:
>>
>>> On Mon, May 23, 2011 at 01:15:31PM +0200, S Roderick wrote:
>>>> On May 23, 2011, at 05:55 , Theo de Vries wrote:
>>>>
>>>>> I have been searching a bit and saw that some people (Ruben Smits,
>>>>> Philippe Hamelin?) used RTT with Linux and the rt preempt patch. We
>>>>> are planning to do the same.
>>>>> As far as I understand from this page
>>>>> (https://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO), we have to do
>>>>> the following:
>>>>>
>>>>> 1. Setting a real time scheduling policy and priority.
>>>>
>>>> RTT takes care of this for you. Set an activity to ORO_SCHED_RT and the period and priority of your choice.
>>>>
>>>>> 2. Locking memory so that page faults caused by virtual memory will
>>>>> not undermine deterministic behavior
>>>>
>>>> Well, actually locking prevents paging at all ... which stops the non-deterministic behaviour.
>>>
>>> Do you mlockall in configureHook? (This should be be a deployment
>>> property, shouldn't it)
>>
>> Yes, and yes it really should be.
>
> Ok.
>
>> Also, remember that mlockall() alone isn't sufficient - you also
>> have to set some malloc options (see the RT Wiki page referenced
>> above, IIRC).
>
> Not sure which options you mean here...
We have the following (error checking, etc, elided)
mlockall(MCL_CURRENT | MCL_FUTURE)
mallopt(M_TRIM_THRESHOLD, -1)
mallopt(M_MMAP_MAX, 0)
orocos rtt with rt preempt
On Mon, May 23, 2011 at 01:43:10PM +0200, Stephen Roderick wrote:
> On May 23, 2011, at 07:41 , Markus Klotzbuecher wrote:
>
> > On Mon, May 23, 2011 at 01:35:25PM +0200, Stephen Roderick wrote:
> >> On May 23, 2011, at 07:25 , Markus Klotzbuecher wrote:
> >>
> >>> On Mon, May 23, 2011 at 01:15:31PM +0200, S Roderick wrote:
> >>>> On May 23, 2011, at 05:55 , Theo de Vries wrote:
> >>>>
> >>>>> I have been searching a bit and saw that some people (Ruben Smits,
> >>>>> Philippe Hamelin?) used RTT with Linux and the rt preempt patch. We
> >>>>> are planning to do the same.
> >>>>> As far as I understand from this page
> >>>>> (https://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO), we have to do
> >>>>> the following:
> >>>>>
> >>>>> 1. Setting a real time scheduling policy and priority.
> >>>>
> >>>> RTT takes care of this for you. Set an activity to ORO_SCHED_RT and the period and priority of your choice.
> >>>>
> >>>>> 2. Locking memory so that page faults caused by virtual memory will
> >>>>> not undermine deterministic behavior
> >>>>
> >>>> Well, actually locking prevents paging at all ... which stops the non-deterministic behaviour.
> >>>
> >>> Do you mlockall in configureHook? (This should be be a deployment
> >>> property, shouldn't it)
> >>
> >> Yes, and yes it really should be.
> >
> > Ok.
> >
> >> Also, remember that mlockall() alone isn't sufficient - you also
> >> have to set some malloc options (see the RT Wiki page referenced
> >> above, IIRC).
> >
> > Not sure which options you mean here...
>
>
> We have the following (error checking, etc, elided)
>
> mlockall(MCL_CURRENT | MCL_FUTURE)
Yes, this is standard preamble for realtime programs.
> mallopt(M_TRIM_THRESHOLD, -1)
> mallopt(M_MMAP_MAX, 0)
IIRC you only need these if you do want to do dynamic allocations in
realtime.
Markus
orocos rtt with rt preempt
On May 23, 2011, at 05:55 , Theo de Vries wrote:
> I have been searching a bit and saw that some people (Ruben Smits,
> Philippe Hamelin?) used RTT with Linux and the rt preempt patch. We
> are planning to do the same.
> As far as I understand from this page
> (https://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO), we have to do
> the following:
>
> 1. Setting a real time scheduling policy and priority.
RTT takes care of this for you. Set an activity to ORO_SCHED_RT and the period and priority of your choice.
> 2. Locking memory so that page faults caused by virtual memory will
> not undermine deterministic behavior
Well, actually locking prevents paging at all ... which stops the non-deterministic behaviour.
> 3. Pre-faulting the stack, so that a future stack fault will not
> undermine deterministic behavior
We actually never do this (as its actually quite hard to do), though you are supposed to. We also never explicitly pre-fault any large data structures (as you are supposed to), as given that most of our data structures are complex objects their constructors take care of that for us. Despite this, we still run in real-time.
All the above are required for real-time performance, however, there are other necessary things you must take care of yourself, for examle
- don't dynamically allocate memory (unless you are using TLSF) once you are into the real-time part of your application
- don't use RTT::Logger (its not realtime)
- don't write to disk in a real-time component.
- etc.
We monitor our "realtime-ness" (in part) by tracking major and minor page faults on either a per-thread or per-process basis (which ever makes sense given the application). At least that way you have data telling you whether you stayed in real-time or not. NB this data isn't sufficient to tell the whole story, but it captures most problem cases.
HTH
S